- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Summary
The paper explores the question of where in a neural network should a utility-dependent gradient modulation be applied to preserve plasticity. It compares three kinds of gating: output-only, hidden-only and full-network. The motivation behind this question is the observation that most plasticity-preserving methods are generally applied across the whole network, but different distribution shifts might require different parts to adapt. They use an already established method UPGD-W and provide both theoretical analysis and empirical results in a variety of benchmarks. Making gating location an experimental design variable has a potential positive impact in the literature.
Comment to authors
Overall, I found this paper to be strong and thoughtful. The research question is clear and the ablations and main empirical results are sufficient to support the main claim. My main concerns are about robustness and scope rather than the main idea. I think that the discussion should better incorporate recent work, and better place itself around those. If these points are addressed satisfactorily, I would be comfortable increasing my score.
Critical strengths
Overall, I believe the paper has potential for impact in the community, and that the strengths outweigh the weaknesses.
- Clear research question. And clear experimental ablation construction with the isolation of gating placement from other types of parameter freezing.
- Strong empirical results on label-perm. The main empirical finding is clear rather than marginal.
- Extensive and qualitatively different shift benchmarks. The results are more nuance than just a single simple claim. There is a difference between label permutation results (favoring output-only) and Gridworld (favoring hidden-only), and that points to the idea of properly placing gating based on the adaptation dynamics.
- Proper and deeper mechanistic analysis. I appreciate the authors going deeper into the diagnostics and checking whether the performance differences are consistent with what the actual mechanism proposes rather than just an artifact of some other obscure interaction.
- Honesty about limitations. I appreciate the intellectual honesty about the theory and claims made in the paper such as the checkpoint displacement being only a heuristic proxy or that theory does not provide a universal gating-placement rule.
Critical weaknesses
Major
-
The strongest result for supervised learning are biased for output-layer. The 3 label-permutation tasks provide an input that is fixed and only change the mapping classes->output labels, thus it might logically follow that output-only gating performs much better. Including an end-to-end representation shift benchmark like domain shifts or purely class-incremental learning with new classes would strengthen the claims of the paper.
-
The statistical results might be biased. The main comparisons should be re-run with matched independent seeds. If this was the case, please indicate so.
-
Gridworld results should be verified under a fixed nº of env interactions or same nº of optimizer steps. If this was the case, please indicate so and/or consider it a necessary ablation to make robust claims about such results.
Minor
-
Incomplete coverage of recent plasticity related literature. There are related-work omissions that are directly relevant to the mechanisms studied while interpreting layer-wise plasticity. First of all, Lyle et al. [1] argues that plasticity loss arises from many mechanisms interacting at the same time rather than a single property, and that needs to be take into account for the papers interpretation. Also, Lillo & Cheney [2] showed that the activation function of choice strongly affects plasticity in supervised CL and RL, and that is relevant to the authors work as they fix ReLU while drawing conclusions about hidden-layer adaptability. Park et al. [3], and Rohani et al. [4] also studied activation dynamics and increased linearity as mechanisms for maintaining plasticity, and that ties into the study of dormant units and representation plasticity. Finally, Rosseau et al. [5] introduces AdamO, which is an Adam-style optimizer that decouples isometry-promoting regularization from the adaptive gradient update, thus this should also be discussed as a recent optimizer-based approach to maintaining plasticity.
-
The gated parameter fraction (0.16%-2.5%) might be misleading. While output-gating acts only on those small %s, the utility still tracks over all parameters. The main text should make this distinction clear.
-
A more careful statement about GridWorld dynamics-shift. Adam achieves basically the same return and that should be explicitly stated and discussed.
-
The replay-buffer ablation in App. I.1 weakens generalization of GridWorld results. I'd consider discussing this in the main text and exploring its consequences as the results become closer in absolute value.
References
[1] Lyle et al., “Disentangling the Causes of Plasticity Loss in Neural Networks” (CoLLAs 2025)
[2] Lillo & Cheney, “Activation Function Design Sustains Plasticity in Continual Learning” (ICLR 2026)
[3] Park et al., “Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss” (ICML 2025)
[4] Rohani et al., “Preserving Plasticity in Continual Learning with Adaptive Linearity Injection” (CoLLAs 2026)
[5] Rosseau et al., “Preserving Plasticity in Continual Learning via Dynamical Isometry” (ICML 2026)
Questions for authors
-
Based on Theorem 3.2, could the authors empirically measure displacement-based utility and examine its relationship with gated curvature?
-
Do the authors recognize if the layer-placement effect might persist in other shifts? Like in domain shifts, class-incremental shifts, or end-to-end CNNs (see Critical Weaknesses Major 1).
-
Could GridWorld be matched by env transitions or optimizer updates rather than episodes? (See Critical Weaknesses Major 3).
-
Could the authors clarify if there is a multiplier for non-gated parameters in the RL experiments? If so, what made that specific value to be chosen? There should be exploring those dynamics.
4 comments
Task design, efficiency and related work (Part 2 of 2)
Task design and generalization (Major W1, Q2)
We follow the streaming input- and label-permutation benchmarks of Elsayed and Mahmood (ICLR 2024), Sections 4.2–4.4 and Figures 3 and 6. Dohare et al. (Nature 2024), Methods, “Robust loss of plasticity in permuted MNIST,” and Extended Data Figure 3, study input-permuted MNIST one example at a time. Lyle et al. (2025), Section 3.1 and Figure 1, study plasticity loss under repeated label randomization. We keep the architecture and training conditions fixed across gating variants to compare placement directly.
For label permutations, both Full and Output gate the output layer. Their comparison tests whether excluding hidden layers from adaptive gating improves adaptation when useful features can be reused. Hidden layers continue to learn even when they are not adaptively gated. Neither label nor input permutation mathematically determines the best gating placement. Output has higher mean accuracy than Full on the three label-permutation benchmarks, while the observed means are close on Input-Permuted MNIST. These results support studying gating placement across different learning settings.
Efficiency potential and current cost (Minor W5)
The gated-parameter fraction does not by itself measure computational savings. We benchmarked the same Mini-ImageNet MLP and optimizer settings on one CPU thread in FP32, using batch size one and five repetitions of 1,000 training updates after warm-up. Output took 3.58 ms per update compared with 4.39 ms for Full, about 18.5% less time. Both retained the same 2.7 MB utility state because utility tracking still covers every parameter. Removing unused utility tracking from Constant-0.5 reduced update time from 3.56 to 3.07 ms without changing its training trajectory. These measurements support a runtime benefit for output-only gating in this implementation, while reducing utility storage remains future work. We will revise the abstract to distinguish the placement finding from computational savings.
Displacement utility and gated curvature (Q1)
Appendix E.2, Figure 4, and Table 6 report Spearman correlations of utility magnitude with diagonal Hessian magnitude and squared gradients across supervised learning, Gridworld, and MuJoCo. Across the reported settings and methods, mean correlations range from approximately 0.22 to 0.98 for curvature and 0.76 to 0.99 for squared gradients. These measurements show that the utility used to set our gates is positively associated with local curvature, with stronger correlations to squared gradients. They support the utility–curvature connection for the implemented optimizer, although they do not directly test the displacement-based relationship in Theorem 3.2.
Related work and the interpretation of plasticity (Minor W4)
We thank the reviewer for this suggestion. We will discuss how the papers you suggested relate to gating placement in the related-work section of the main text. We keep the architecture and update rule fixed to examine where utility gating is useful. Future work could test whether activation design changes which layers benefit from utility gating.
We will share all experiment and analysis code along with the full hyperparameter settings.
Thank you to the authors for your rebuttal. Most of my concerns have been addressed and I believe the paper provides interesting and relevant results to the continual learning community.
However, I do not see the new revised version as a the pdf submission. Could the authors update the pdf submission of the paper with the new revised version before the rebuttal process ends?
If that is correctly done I will be keeping my Weak Accept (3) rating and I will argue for acceptance of this paper.
Thank you very much for reviewing our responses and for your support. We appreciate the reminder to update the submission PDF. We are incorporating the corrections, new experiments, and clarifications into the manuscript and will upload the revised version before the discussion deadline. We will let you know here as soon as it is available.
Gridworld and matched comparisons (Part 1 of 2)
Gridworld budgets and adaptation (Major W3, Minor W6, Q3, Q4)
We thank the reviewer for these suggestions. Rebuttal Table 2 in our response to Reviewer qNrp shows the new Gridworld results. The submitted runs used equal episode counts. The new experiments instead fix both interactions and optimizer updates, giving every method 500,000 interactions and 499,936 updates per run across all three shifts, with ten paired seeds per condition. For the question about nongated parameters, nongated layers multiply the normalized gradient plus injected noise by 0.5, with weight decay applied separately. We use a fixed multiplier of 0.5 because the sigmoid-based gating function returns 0.5 at zero normalized utility.
We agree that the original dynamics-shift results did not establish an advantage of Hidden over Adam. In the new experiment, mean early success was 90.31% for Hidden, 90.05% for Constant-0.5, and 87.78% for Adam. Together with the other shift conditions, these results support our finding that the usefulness of gating placement depends on the learning setting.
Replay and the scope of the finding (Minor W7)
We thank the reviewer for raising this concern. We repeated all three shifts with replay, using five methods and ten paired seeds per condition. Batch size remained one, with 500,000 interactions and 499,936 updates per run. The 10,000-transition buffer was retained across shifts.
Rebuttal Table 3. Success (%) without / with replay, averaged over ten seeds and nine post-shift phases. Success means reaching the goal within 200 steps. Early/final cover the first/last 10K interactions of each 50K phase. Bold marks the highest averages separately without and with replay; comparisons are descriptive.
Under dynamics shifts, Hidden and Constant-0.5 both reached 80.66% mean early success, compared with 74.46% for Output. Replay reduced final success for every method under goal and joint shifts and changed the placement ordering. We will discuss this dependence on replay in the main text. Our interpretation is that replaying rewards from the old goal interferes with adaptation to the new goal. This early interference may hinder exploration and learning even after the old experiences leave the buffer.
Independent seeds and matched comparisons (Major W2)
We checked all 28 entries in Table 1 and recalculated the statistics across distinct seeds. Counting the same seed three times caused the zero SE for Full and Output on CIFAR-10. The corrected accuracies are 59.96 ± 0.43% and 61.71 ± 0.51%, respectively, across ten seeds. Input-MNIST S&P now uses five seeds, giving 78.29 ± 0.15%. The corrected Table 1 lists all seed counts and optimizer settings. To address your request for matched independent seeds, we ran a new Mini-ImageNet comparison with Full, Output, and Constant-0.5 across five independent seeds. Within each seed, initialization, example order, label permutations, noise streams, learning rate, and decay were matched across methods. Output exceeded Full by 7.51 percentage points, with a 95% confidence interval of 6.73–8.29. The new Gridworld comparisons use ten paired seeds per condition. Table 1 was corrected from existing runs; the new matched experiments cover Mini-ImageNet and Gridworld.