NAMI: A Heuristic Forward Transfer Approach for Continual Reinforcement Learning

Continual reinforcement learning (CRL) requires a reinforcement learning agent to adapt rapidly to new tasks by reusing knowledge acquired from previous ones, a capability known as forward transfer. Existing architecture-based methods, which typically maintain task-specific modules and reuse them for future tasks, often perform forward transfer at the level of entire networks. While effective, such coarse-grained transfer entangles reusable and task-specific modules, making it difficult to reuse prior knowledge flexibly under strict interaction budgets. In this paper, we propose Neuron-wise mAnhattan-norm Maximization Initialization (NAMI), a simple and efficient strategy for forward transfer in CRL. Instead of transferring the entire policy networks, NAMI initializes each neuron of a new task policy network by selecting the corresponding neuron with the largest Manhattan norm (i.e., the maximum sum of absolute weights) from the previously learned policy networks. This converts model-level transfer into neuron-level selection, allowing the new policy to inherit salient reusable components from different prior tasks while remaining a single compact network whose inference cost is independent of the number of previous tasks. Extensive experiments show that NAMI achieves superior performance over various state-of-the-art methods.