Hierarchical planning with cross-level credit assignment for long-horizon tool use

Long-horizon tasks require large language model (LLM) agents to plan hierarchically, in which a global plan fixes task-wide objectives and local subgoals adapt to observations revealed only during execution. When to place these local subgoals depends on observations that emerge during execution, so that we formulate it as a learning problem which however poses a credit assignment challenge. Prior hierarchical planning methods supply this credit through a learned value model or a prescribed process reward, which provides neither the after-execution evidence the global plan needs nor the task-wide reference a local subgoal lacks. We introduce HiCLA (Hierarchical Planning with Cross-Level credit Assignment), which supplies both through cross-level on-policy self-distillation: after a trajectory terminates, the policy (1) generates terminal evidence that rescores the global plan (i.e., execution-to-planning) and (2) produces a revised plan that in turn rescores each recorded state and decision (i.e., planning-to-execution), supplementing GRPO and used only during training. Across three Qwen backbones, HiCLA achieves the highest τ²-bench mean success over four evaluation seeds among the compared methods and exceeds the strongest baseline on held-out BFCL-V3 with the 7B and 4B models by up to 5.25%.