acceptodds
Under review as a conference paper at ICLR 2027

NEUROPLASTIC: A PLASTICITY-MODULATED OPTIMIZER FOR BIOLOGICALLY INSPIRED LEARNING DYNAMICS

Abstract

Optimization algorithms are fundamental to modern deep learning, yet most widely used methods rely on update rules based primarily on local gradient statistics. We introduce NeuroPlastic, a plasticity-modulated optimizer that aug- ments gradient-based updates with an adaptive multi-signal modulation mech- anism inspired by multi-factor synaptic plasticity, a concept from neurobiol- ogy. NeuroPlastic dynamically scales gradient updates using interacting com- ponents that capture gradient, activity-like, and memory-like statistics, forming a lightweight modulation layer compatible with standard deep learning training pipelines. Across image classification benchmarks, NeuroPlastic consistently im- proves over a controlled gradient-only ablation, with more pronounced gains on the Fashion-MNIST benchmark and in reduced-data regimes. In transfer experi- ments on CIFAR-10 with ResNet-18, the method remains stable and competitive without retuning. These results suggest that multi-signal plasticity-inspired mod- ulation can provide a useful extension to conventional gradient-driven optimiza- tion, particularly when learning signals are limited or noisy, and offer a promis- ing direction for gradient-based methods in deep learning.

Primary area:
optimization
Keywords:
Machine Learning, Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_jqajOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes NeuroPlastic, an optimizer inspired by multi-factor synaptic plasticity in neurobiology, which attempts to introduce a multi-signal modulation mechanism into backpropagation to achieve adaptive gradient scaling. The paper uses gradient magnitude, moving averages, and a memory-like state to compute a modulation factor, thereby stabilizing network parameters. On small-scale datasets and in data-scarce settings, NeuroPlastic achieves slight improvements over established optimizers such as SGD and Adam, but under more complex experimental conditions (CIFAR-10 + ResNet-18), NeuroPlastic performs worse than the tuned baseline optimizers.

Critical strengths
  1. Translating the metaphor of multifactor interactions in synaptic plasticity into adaptive modulation factors for gradient flow is an interesting design idea that provides a heuristic approach to dynamic gradient scaling.

  2. The motivation is presented clearly, the structure is generally complete, and the mapping from neurobiological concepts to optimizer components is explained intuitively.

Critical weaknesses

1.The synaptic plasticity described in the paper is essentially a nonlinear combination of gradient moving averages and second moments, rather than actual local or global neuronal activity and neuromodulatory behavior involved in synaptic plasticity. Compared with existing baseline optimizers, it lacks a breakthrough at the mechanistic level.

2.The experimental results presented in the paper indicate that the proposed optimizer lacks competitiveness. Furthermore, compared with the widespread application of classical optimizers, NeuroPlastic lacks experiments on large-scale tasks.

3.NeuroPlastic maintains three parameter-sized state tensors, requiring approximately 50% more optimizer-state memory than AdamW at equal precision, as acknowledged in Appendix A.1. Consequently, it remains unclear whether the modest improvements over the gradient-magnitude-only ablation justify the additional computational and memory costs.

4.NeuroPlastic introduces multi-factor computations and bounded stabilization steps, involving several new decay rates, smoothing constants, and clipping thresholds, but the paper provides no analysis or experiments concerning these parameters.

Questions for authors

I recommend conducting benchmark evaluations on at least ImageNet-1k image classification and small-scale language models. If the method only provides benefits on small-scale tasks, its practical value for modern machine learning would be considerably limited.

Reviewer_x2XhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces a neuro-inspired optimiser that scales the gradients using a combination of current gradient magnitude, a moving average and first and second moment statistics. The authors evaluate their proposed method on MNIST, Fashion-MNIST etc.. and observe marginal improvements or in some cases performance degradation compared to established optimisers.

Critical strengths
  • The optimizer is explicitly defined and can be implemented within standard gradient-based training.
  • The neuro-perspective could be relevant for some audiences.
  • The paper reports unsuccessful transfer results and discusses its limitations.
Critical weaknesses
  • The experiments do not establish a consistent improvements over standard optimizers.
  • The paper introduces several modulation and stabilization components without isolating their fully individual effects.
Questions for authors
  • Could you consider comparing NeuroPlastic with newer optimizers such as Muon?
  • Are there specific tasks or data regimes where NeuroPlastic offers a consistent advantage over standard optimizers?
Reviewer_skGnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes NeuroPlastic, an optimizer that scales each gradient coordinate by a coefficient built from three gradient-derived statistics (current gradient magnitude, an EMA of gradient magnitudes, and an Adam-style ratio), then applies a norm-clipping and gain-based stabilization step. The design is loosely motivated by multi-factor synaptic plasticity. On MNIST and Fashion-MNIST, the full rule gives small gains over a gradient-magnitude-only ablation. On CIFAR-10 with ResNet-18, it trails tuned SGD, Adam and AdamW.

Critical strengths

1- The method is described precisely, and the authors are open about implementation details such as the missing bias correction

2- The experiments section and conclusion are honest. They don’t overstate the results, and they acknowledge the CIFAR-10 gap and the limited tuning parity.

3- The ablation is cleanly defined, code is provided, and the negative design-space results (Fig. 7) are a useful inclusion.

Critical weaknesses

1- The biological connection is very loose.\textbf{The biological connection is very loose.} In three-factor plasticity, the Hebbian or eligibility term is local: it depends on pre- and post-synaptic activity at the synapse. Here every signal is derived from the backpropagated gradient, which already carries global credit-assignment information. The per-tensor normalization adds another non-local step on top of that. The authors concede most of this themselves. It seems to me that the work is grounded neither in theoretical analysis nor in strong biological observations, and its insights are limited to a heuristic modification of SGD.

2- A combination of familiar statistics with no theory and little positioning.\textbf{A combination of familiar statistics with no theory and little positioning.} Each ingredient is standard. In fact, ∣m∣v\frac{|m|}{\sqrt{v}} is just the magnitude of the Adam step. The paper gives no analysis of what combining them does: no convergence result and no look at the implied effective step size. Some design choices also go unexplained

(a): Eq. 14 amplifies large-gradient coordinates, the opposite of what Adam-style preconditioning does. (b): The update direction is the raw gradient rather than mtm_t, even though mtm_t is computed. (c): At the default settings (ρ=0.01\rho=0.01, rtarget=0.02r_{target} = 0.02), the “homeostatic” gain stays within about 1% of 1. (d): Related work that does similar things is missing from the discussion

3- Weak results and limited experiments.\textbf{Weak results and limited experiments.} On the two benchmarks where gains are reported (MNIST and Fashion-MNIST), there are no standard baselines. The absolute accuracies look low for these tasks. On CIFAR-10, the method is clearly behind the baselines and quite sensitive to the learning rate. The abstract and contributions (“consistently improves”, “scales with data scarcity”, “competitive without retuning”) claim more than the results in Sections 4–5 support. Overall, experiments and results are very limited.

Other Issues:

  • The paper uses the ICLR 2026 template.
  • The link to the GitHub repository might compromise anonymity.
Questions for authors

Please refer to weaknesses section

Comments on the paper

1 comment
Area_Chair_A1B4OpenReview ↗

Reviewer's responses to authors

Dear reviewers,

If not done already (thank you - rev. x2Xh), could you please take a look and reply briefly to the "Official Comment(s) by Authors" / Rebuttals, possibly by an acknowledgement that you have seen it.

Feel free to respond in more detail, in particular, if you have further questions or opinions to the replies from the authors.

With best regards, an AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.