The Effects Of Volumetric Morphology Encodings On Online Universal Control

Universal control methods tackle the problem of learning to control diverse robot morphologies from online interactions, using reinforcement learning. Recent works have demonstrated that proper conditioning on morphological context is key to obtaining good performance across these incompatible Markov decision problems, i.e. problems with varying input and output spaces. In this work, we identify the robot's kinematic context as a particularly important portion of the overall contextual information. To provide the body shape portion of the kinematic context to universal controllers, existing methods have leveraged the strong assumption that robot bodies are composed of simplified geometric primitives and have directly included the descriptors of those shapes in the context vector. In this paper, we propose volumetric morphology encodings (VME). These novel representations capture the body shape and connectivity of the robot, and are learned during a pretraining phase, from a 3D robot body shape dataset that we collect. We use VME embeddings to improve the training of universal controllers, on challenging online universal control tasks. Even if we only alter the body shape portion of the kinematic portion of the overall robot context, we observe substantially higher performance than baseline methods, with higher gains in harder tasks. Furthermore, our approach demonstrates superior zero-shot transfer to unseen embodiments.