Split the World: Are Regional Dynamics Better Than Global Dynamics for Latent World Models?

Latent world models, due to their efficiency, have become one of the main paradigms for world modeling. In latent space, they learn from data the dynamics governing how an agent’s actions affect the environment and use these dynamics to predict the environment state after executing actions. Traditionally, most existing latent world models learn a single global dynamics model from the full task dataset, despite possible differences in dynamics across different regions of the task state space. Existing approaches have shown that regional dynamics can be useful. However, either global or regional dynamics are used by default, without a clear and general theory for selecting between them. Motivated by this, we study when regional dynamics are better than global dynamics and formulate this question as a dynamics-selection problem. To answer this question, we propose the Worst-Case Partition Gain criterion, which is based on Wasserstein distances between regional response distributions. Using this criterion, we establish a necessary-and-sufficient condition for selecting the better dynamics under reasonable assumptions. Furthermore, we develop a dynamics-selection algorithm based on this condition, with a calibrated threshold held fixed across different modeling configurations. We evaluate our method using, to our knowledge, the first systematic experimental framework for dynamics selection, comparing 22 methods on transfer across different numbers of regions and world-model backbones. Experimental results demonstrate state-of-the-art selection performance: our method correctly selects the better dynamics in all 8 region-count transfer configurations and in 18 of 20 experimental configurations overall.