Winner-Take-All bottlenecks enforce disentangled symbolic representations in multi-task learning

Winner-take-all (WTA) networks constitute a central circuit motif in cortical networks of the brain. In addition, WTA-like activations are abundant in modern deep learning models in the form of the softmax activation for example in attention layers of transformers. While their role in the extraction of latent factors has been studied for relatively simple generative models, their role in the context of highly non-linearly entangled latent factors has remained elusive. In this article, we show that a WTA bottleneck within a deep neural network can enforce under certain well-defined conditions the extraction of categorical latent factors of the data in a multi-task learning setup. In particular, we prove that under specific conditions, including that the structure of the WTA bottleneck matches the structure of the latent factors and the model solves all tasks perfectly, the emerging representation is a structured permutation of those latent factors. When the structures do not match, we empirically demonstrate on two datasets that a symbolic representation often emerges when the multi-task learning objective is optimized close to its global optimum. This representation is a weaker form of structured permutation in which multiple WTA output neurons may jointly encode a single latent factor category. We additionally evaluate the generalization behavior of the acquired symbolic representation. Our proposed model provides insights into the generalization capabilities of deep neural networks with WTA-like components and may serve as an interface between symbolic and subsymbolic AI systems.