Matryoshka World Models: Test-Time Scaling Beyond Search

Planning with learned world models requires trading off predictive fidelity against computational cost. Existing approaches typically vary search effort while keeping the latent representation fixed, or train separate models for different representation sizes. We introduce Matryoshka World Models (MWM), a framework for training world models to operate at multiple nested predictive fidelities. Unlike independently trained fixed-dimensional models, MWM learns a shared hierarchy of predictive representations that can be truncated or expanded at test time without retraining. This variable-fidelity world model enables a new form of planning-time flexibility: predictive fidelity can change across episode-level replanning steps, CEM optimization iterations, and rollout depth. The planner can therefore allocate high-fidelity computation selectively while using cheaper representations where fine-grained prediction is unnecessary. We evaluate variable-fidelity planning against standard fixed-fidelity planning baselines across navigation and manipulation tasks. MWM matches the success rate of the full-fidelity baseline at up to 108.8x less compute and exceeds the best baseline by up to 59% at the same compute budget. By making predictive fidelity a resource that can be dynamically allocated across different stages of planning, MWM establishes a new axis of test-time scaling for world models. Project website: https://study-9f2c7a.pages.dev.