Controlling World Model Generation for Imagined Curriculum Learning

Recent work has begun to explore performing Unsupervised Environment Design (UED) within learned world models, aiming to enable adaptive training curricula without requiring continued access to the true environment. However, existing approaches provide limited control over the trajectories generated by the world model. In this work, we propose an improved method of controllable world-model-based UED by introducing Dynamically-controlled Model Generation (DyMGen), a world-model architecture that learns latent generation actions through which an adversarial teacher can control world generation dynamically as a student explores the environment. This enables online adversarial environment generation without requiring a handcrafted simulator that explicitly supports this dynamic environment modification. We show that agents trained in adversarially controlled DyMGen worlds significantly outperform those trained with random generation, and approach the performance of UED agents trained with full access to the true environment. Our results demonstrate the potential for learning environment-design interfaces directly from offline experience, illustrating the potential for UED within learned, controllable world models.