BEAT: BOTTLENECK-GUIDED SELF-EVOLUTION OF EMBODIED AGENTS FOR LONG-HORIZON TASKS

Long-horizon embodied tasks require skill organization that matches controller capabilities, yet skill decompositions are often fixed in advance. We introduce BEAT, a bottleneck-guided self-evolution framework that jointly adapts the planner and controller through execution feedback. Starting from a coarse control tree, BEAT identifies bottleneck skills through end-to-end rollouts and first improves their controllers with targeted online reinforcement learning. Fresh rollouts continually refresh skill-entry states, keeping local training aligned with the evolving agent. When local learning plateaus while failures persist, the planner refines skill granularity and task organization through sequential decomposition or conditional branching. Candidate structures are evaluated against the incumbent after controller adaptation using matched end-to-end rollouts. Across 30 Minecraft long-horizon tasks, BEAT achieves 58.9% average success, compared with 51.1% for the strongest evaluated baseline. Targeted controller adaptation further increases average success from 61.5% to 72.9% on a six-task subset. In Craftax, where controllers are trained from scratch, BEAT achieves 23.93% average success across four goals within an 800M-step budget, compared with 20.41% for the strongest evaluated baseline. These results support execution-driven adaptation of skill organization and controller capabilities for long-horizon tasks.