Stagnant Neurons: Understanding and Restoring Plasticity in Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) methods can suffer from loss of plasticity, gradually becoming harder to adapt under sequential task changes. We identify stagnant neurons as a recurring neuron-level pattern associated with im- paired adaptation. In value-factorization methods, these units exhibit disproportion- ately small gradient-based update activity relative to parameter scale and layer-wise peers. Existing plasticity interventions are not designed to identify and repair this pattern directly. We therefore propose Knowledge-retentive Neuron-level PlastIcity Focusing InjEction (KNIFE), a diagnosis-conditioned, neuron-local plasticity re- allocation method that exactly preserves the network function at the intervention boundary. RUA identifies impaired units, local surgery restores trainable capacity, and decay with pruning recycles temporary capacity. Across SMACv2, SMAC, Predator–Prey, matrix games, and MEAL, KNIFE consistently outperforms the compared plasticity interventions. Beyond value factorization, KNIFE also im- proves adaptation with MAPPO, MADDPG, and DGN, spanning policy-gradient, actor–critic, and graph-based MARL learners. Additional single-agent PPO results demonstrate that the intervention is not specific to value-factorization training.