Recursive Successor Models

Planning with world models enables improvement beyond the dataset, but compounding prediction errors hinder long-horizon planning. To prevent one-step compounding error, we instead model longer-term future state distributions, known as successor measures. We introduce Recursive Successor Models, which combine (1) a stable supervised training objective that avoids bootstrapping with (2) a recursive sampling procedure for planning that composes short-horizon distributions into long-horizon distributions. We demonstrate the capabilities of RSM in simulated benchmarks and real-world robot experiments by showcasing improvements in policy performance, planning latency, and training convergence speed across long-horizon and visual tasks.