Joint Learning of Hierarchical Neural Options and Abstract World Model

Building agents that can perform new skills by composing existing skills is a long-standing goal of AI agent research. Towards this end, we investigate how to efficiently acquire a sequence of skills, formalized as hierarchical neural options. However, existing model-free hierarchical reinforcement algorithms need a lot of data. We propose a novel method, which we call "AgentOWL" (Option and World model Learning), that jointly learns — in a sample efficient way — an abstract world model (abstracting across both states and time) and a set of neural options. We show, on a subset of Object-Centric Atari games, that our method can learn more skills using less data than baseline methods and possesses learning and generalization capabilities that the baselines do not have.