Simple Neural Techniques for playing games of imperfect information

Developing intelligent policies for partially observable environments is challenging, as it requires reasoning about uncertainty over hidden states, planning actions under uncertain beliefs, valuing information acquisition and concealment, and finding robust policies that cannot be easily exploited. Imperfect-information games provide a natural testbed for these capabilities. In this paper, we develop simple, general techniques for efficiently training neural policies for such games, focusing on Fog-of-war chess, a game that combines the strategic complexity of chess with the challenges of imperfect information. Inspired by methods for solving imperfect-information games based on Counterfactual Regret, we adapt these ideas to reinforcement learning with large neural networks. Our approach achieves competitive performance against the state-of-the-art search techniques while doing no inference time compute or search and only requiring only a fraction of the compute used by other regret-based neural policy learning methods.