Complex Reasoning Tasks have Directional Structure

While many datasets are believed to lie near low-dimensional manifolds, many different geometric structures can explain the same observations. What differentiates one manifold from another is its geometry: which inputs are close together and which are far apart. In language, this locality is typically defined by word co-occurrences. As linguist J. R. Firth put it, "You shall know a word by the company it keeps." In vision, locality is often defined via data augmentations: images that differ by a random crop or a small jitter should be similar. However, how to define this locality for downstream decision-making tasks is unclear. Decision-making, is directional: one state can be close to another in the direction of task progression, but not vice versa. Thus, we argue that to represent sequential tasks, a manifold requires a notion of direction in addition to static representations. In this paper, we show how to equip contrastive reinforcement learning with bits about time, regularising the learned representations. We show that this regularisation improves performance on complex reasoning tasks, reducing the number of steps required to solve the Rubik's Cube by 26% and improving learning efficiency.