DeepPlanning-ACE: Can Language Agents Ace Interactive Long-Horizon Planning?

Long-horizon planning benchmarks typically assume that user requirements are complete, unambiguous, and fixed from the outset. Real-world planning, however, is inherently interactive: users disclose constraints gradually, clarify underspecified preferences, revise earlier decisions, and sometimes make requests for which no valid solution exists. We introduce DeepPlanning-ACE, named after capabilities essential to interactive planning: resolving Ambiguity, adapting to Change, and Explaining infeasibility. This also captures the benchmark's central question: can language agents truly ace deep planning as user intent evolves through interaction? DeepPlanning-ACE comprises bilingual episodes derived from verifiable base instances in travel planning and multi-product shopping. It preserves the original tasks and constructs ambiguous, changing, and infeasible variants when their defining conditions can be verified. To enable controlled and reproducible user simulation, the benchmark decouples semantic intent from linguistic realization. User requirements, intent changes, and infeasibility conditions are specified and verified in advance. A deterministic controller governs information disclosure, state transitions, turn limits, and termination, while a language model can select from authorized disclosures without modifying their canonical content. Evaluation covers both task completion and interaction quality along the ACE dimensions: clarifying ambiguous requirements, adapting to changing constraints, and explicitly explaining infeasibility. Experiments across nine language agents show that multi-turn interactive planning remains challenging for all evaluated models, revealing persistent limitations in maintaining valid plans, adapting to changing requirements, and explaining infeasibility with sufficient tool evidence.