Learning from the Unexpected: A Von Economo Neuron-Inspired Efference Copy Architecture for Continual Reinforcement Learning

Efficient continuous learning requires computing the orthogonal subspaces in the learnt parameter space, separating parameters to be updated for the new task from the ones that should be retained. This implies a continuous one-to-one mapping between the input and generated output to distinguish which inputs are caused of the model output, which is also required for animal self-recognition in the mirror. We derived a novel reinforcement learning framework inspired by von Economo neurons (VEN), a type of neuron that is only found in the small number of species with documented mirror self-recognition. Morphology of VEN suggests that it compares efference copy motor plan to the sensory input, helping to yield truly unpredictable residual signals for learning. Unlike the reward-prediction error of standard TD learning, these residual isolate self-caused from externally induced change. We distilled VEN architecture into an explicit efference copy circuit for reinforcement learning. In Atari-2600 game and control suite environments, VEN augmented DDQN learned much more efficiently than capacity-matched controls under self-position perturbation, actively probed their own position in the game, and performed continuous learning across sequentially presented tasks. This architecture provides a computational account of how preserved latent state information could support continuous learning.