Low-Dimensional Decision Spaces for Latent World-Model Planning

Latent world models can make visual planning more efficient by predicting in representation space rather than pixel space. However, conventional methods still predict a separate high-dimensional latent trajectory for every candidate action sequence, even though much visual information is shared across candidates. We study the variation among candidate outcomes for a fixed context, goal, and planning horizon, and refer to it as the *decision space*. Our analysis shows that a small number of directions captures most candidate variation while preserving much of the candidate-ranking signal used for action selection. Based on this finding, we introduce the Low-Dimensional Decision World Model (LDD-WM). LDD-WM performs one shared, action-free visual rollout and uses an action-conditioned component to update a compact $r$-dimensional state for each candidate. The model is trained from physical-cost ordering and avoids reconstructing a separate full latent trajectory for every candidate. Across six datasets, LDD-WM consistently outperforms conventional full-latent reconstruction world models while requiring less computation and lower latency at inference. On MetaWorld Reach-Wall, LDD-WM improves success from $29.9\%$ to $64.6\%$ over JEPA-WM while reducing inference computation by a factor of $150$ and running $71\times$ faster. These results support decision-space modeling as an efficient alternative to full-latent prediction for visual planning.