OPAIR: Objective-Guided Capacity Repopulation for Neuron Recycling

Loss of plasticity limits the ability of deep reinforcement learning agents to keep learning, and neuron recycling mitigates it by periodically resetting under-utilized neurons. Existing methods focus on which neurons to release and reinitialize them with a rule independent of the training objective that zeroes their outgoing weights. The recycled neurons thus receive little learning signal from the objective and return to dormancy before they contribute to reducing it, which leaves the released capacity unused. To enable the released capacity to learn from the objective, we revisit neuron recycling as a two-stage procedure of release and repopulation and propose Objective-guided PAIRed neuron recycling (OPAIR). OPAIR adopts the activation-based release rule of ReDo, which we prove causes only a small objective perturbation, and replaces each pair of released neurons with two copies of an unreleased neuron, chosen by the objective and assigned opposite outgoing weights. This paired construction preserves the network function, and the objective-guided choice maximizes their objective-aligned learning signal along feature-rescaling directions among the candidate donors. Across 27 Atari and MuJoCo tasks, OPAIR achieves the highest mean final return on every task, and mechanism analyses confirm that both the paired construction and the objective-guided selection of copied neurons contribute.