acceptodds
Under review as a conference paper at ICLR 2027

ForesightBench: Causal Evaluation of Prospective Information Acquisition under Irreversible Observability Loss

Abstract

Long-horizon agents take actions that determine both task progress and what remains observable. We study prospective information acquisition under irreversible observability loss: acquiring currently queryable environment state for later use before a task-required transition makes it unrecoverable. We formalize acquisition as constructing a durable sufficient statistic before the acquisition window closes. ForesightBench contains 90 executable, outcome-verified tasks across eight operational domains. Fifty Standard tasks provide lower- and mid-range difficulty anchors, while 40 Frontier tasks introduce longer dependencies, derived state, or multiple information windows. Each Frontier task has a matched Persistent Control designed to preserve the same query interface after the transition while holding other task conditions fixed. Under a common agent scaffold, six agent systems achieve 21.1–41.1% end-to-end success, with 30.0–62.0% on Standard tasks and 2.5–20.0% on Frontier tasks. Across 200 completed matched pairs from five systems, preserving access increases success by 47.5–97.5 percentage points; every system-level contrast is significant after Holm correction (). These results show that the evaluated systems solve many more tasks when future-critical state remains queryable. Identifying which failures arise from missed acquisition windows additionally requires trajectory evidence.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.