Separating Detection from Repair in Long-Horizon Manipulation

A common architecture for long-horizon manipulation pairs a high-level planner with a low-level executor. The planner is expected to notice execution failures and put the task back on course. Research has focused on when to consult the planner, whether at subtask boundaries, on a schedule, or after signs of failure. We test their shared assumption that a planner consulted at the right moment can repair what it finds. Using a monitoring layer built for this study, we compare nine combinations of failure monitoring, planner consultation, and physical retry on identical scenes in RoboCasa365. We evaluate five types of failure signals and find that stalls, or prolonged periods without progress, provide a selective trigger for planner consultation. With ground-truth poses supplied as a diagnostic control, the stall monitor triggers consultations in most episodes that ultimately fail and rarely in those that succeed. Yet detecting trouble does not translate to successful repair. Through systematic experiments and analyses, we find no run-time intervention improves task completion over the executor alone. A review before execution provides a revealing contrast. It corrects known ordering defects and raises completion on some of them, showing that the planner can repair a faulty plan. The persistent stalls we analyze expose a different problem. In half of them, the executor never grasps the object, even though an applicable instruction usually exists. Consulting the planner at a stall does not detectably increase the rate at which the stalled segment resumes, and retrying the same skill rarely succeeds. In the perceptual configuration, the monitors trigger planner consultation in nearly every episode, including successful ones. These comparisons locate the bottleneck in the hands, not the plan. In this setting, the findings point toward training recovery skills for failure states, rather than further changes to when or how the planner is consulted.