Plan2Sense: Open-World Task Planning in Epistemic States via Interleaved Ontic and Sensing Actions

Open-world task planning requires not only generating feasible plans but also acquiring the knowledge needed to execute them. Existing task planners for egocentric agents either attempt to exhaustively gather information from the environment or generate plans under prior assumptions of the world and replan as new observations arrive. Both strategies are impractical in complex environments and can potentially expose agents to undesirable or unsafe situations. To enable effective planning in open-world settings, we propose Plan2Sense, a framework that grounds planning in the agent’s epistemic state to determine what information is required for plan feasibility and how to acquire it. We formalize decision-making as a three-valued first-order Knowledge-State Markov Decision Process (KS-MDP) and develop a knowledge state goal regression algorithm KSR to construct sensing-aware conditional plans in the open-world setting. Our formulation supports sensing actions for both predicate disambiguation and knowledge acquisition via object binding without assuming a fully known initial state or a closed set of predefined objects. Across three open-world egocentric benchmarks requiring active knowledge acquisition, Plan2Sense achieves higher success rates and improved efficiency over existing baselines while consistently avoiding undesirable and potentially unsafe states.