Fast Planning with Cross-Candidate Inference

Sampling-based planning with latent world models typically evaluates every candidate action sequence through a full-horizon rollout. One way to reduce rollout overhead is to fully roll out only a subset of candidates and use the resulting evaluations to reconstruct those of the remaining candidates. The key challenges are to model evaluation correlations among candidates and identify the full-horizon rollouts that are most informative for planning updates. To this end, we propose an efficient planning method for frozen world models that combines cross-candidate inference with rollout allocation guided by action-distribution updates. Using short-horizon predicted states, evaluation trends, and remaining control inputs, the method captures correlations across candidates through a Gaussian process with a trend term. We combine the posterior covariance from this Gaussian process with the local sensitivity of action-distribution updates to derive an update-risk criterion for selecting candidates for full-horizon rollouts. The resulting evaluations are then used to condition the Gaussian process and reconstruct the remaining candidates’ long-horizon evaluations. Observed and reconstructed evaluations then jointly inform the action-distribution update. Experiments across four latent world models and diverse control tasks demonstrate that our method substantially reduces planning latency while maintaining planning performance.