acceptodds
Under review as a conference paper at ICLR 2027

Neuromodulated Equilibrium Learning: Reward-Driven Learning from a Single Free Equilibrium

Abstract

Equilibrium Propagation (EP) is a biologically motivated alternative to backpropagation but requires free and nudged equilibrium phases, raising questions about the biological plausibility of phase-separated learning. We introduce Neuromodulated Equilibrium Learning (NEL), a single-equilibrium reward-based framework inspired by Attention-Gated Brain Propagation and neuromodulatory signaling. After reaching a free equilibrium, NEL propagates a reward-dependent modulatory signal derived from the selected action through reciprocal feedback connections to drive local synaptic updates, eliminating both the reward-nudged equilibrium and contrastive update. We evaluate NEL on MNIST, Fashion-MNIST, and CIFAR-10 using multilayer perceptrons and convolutional networks, with symmetric three-phase REP and standard EP as multi-phase baselines. NEL approaches three-phase REP in shallow architectures, with gaps of 0.79 and 1.31 percentage points on MNIST and one-hidden-layer Fashion-MNIST, while larger gaps of 3.13 and 6.21 percentage points emerge in the deeper Fashion-MNIST and CIFAR-10 settings. A direct cost feedback control using the full supervised output error also remains below three-phase EP, indicating that sparse reward information alone does not explain the performance gap between single-equilibrium and multi-phase learning. These results show that effective reward-driven learning can arise from a single free equilibrium while suggesting that computations introduced by additional nudged relaxations become increasingly important in more challenging architectures.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Equilibrium Propagation, biologically plausible learning, neuromodulation, reward-based learning, credit assignment, energy-based models, synaptic plasticity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KE4NOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Neuromodulated Equilibrium Learning (NEL), a reward-based local learning rule that uses a single free equilibrium. After selecting an output action, a scalar reward generates a modulatory signal that propagates through transposed forward weights and combines with local neuronal activities to update synapses. NEL eliminates the additional nudged equilibria and contrastive updates used by EP and REP. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 demonstrate successful reward-feedback classification, with increasing performance gaps in more complex architectures.

Critical strengths
  • Clear research question. The paper investigates whether equilibrium networks can learn from scalar rewards without additional reward-nudged relaxation, addressing a specific limitation of phase-separated learning.

  • Informative controls. DCF replaces sparse reward feedback with full supervised output error, while BPTT provides a gradient-based reference without additional nudged equilibria. These comparisons help distinguish supervision limitations from deficiencies in credit propagation.

  • Useful empirical characterization. Results cover six random seeds, with analyses of exploration, relaxation length, and update-direction alignment. The paper also acknowledges its performance limitations and does not claim mathematical equivalence to EP.

Critical weaknesses
  • Performance deteriorates with architectural complexity. The gap to three-phase REP increases from 0.79 percentage points on MNIST to 3.13 points on the deeper Fashion-MNIST network and 6.21 points on CIFAR-10. This limits evidence that the proposed credit-assignment mechanism scales effectively. The weak alignment with BPTT in early layers identifies a potential limitation but does not establish its cause or provide a solution.

  • A practical performance–cost advantage is not demonstrated. Using one equilibrium still requires many recurrent updates: MNIST uses 80 relaxation steps, and shortening relaxation substantially reduces accuracy. Without wall-clock, memory, or energy measurements, fewer equilibrium phases alone do not establish greater efficiency, particularly given the accuracy trade-off.

  • Relaxation budgets are not consistently matched. On CIFAR-10, NEL uses 80 free-relaxation steps, whereas REP, EP, and BPTT use 120. Consequently, performance differences mix the effects of the learning rule with differences in free-relaxation computation. Matched-budget comparisons are needed to isolate the contribution of eliminating nudged phases.

Questions for authors

See weakness

Reviewer_N4PdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NEL: it retains the free phase (NEL's free dynamics are Eqs. (23)(24), with the reciprocal feedback strength set to γ=1.0\gamma=1.0), drops the reward-nudged phase and the contrastive update, adopts the action selection and binary reward of BrainProp (Pozzi et al., 2020) to obtain mL=(r−ya0)eam_L=(r-y_a^{0})e_a, then propagates this signal explicitly, layer by layer, through the transposed feedback weights, ml=Wl+1⊤ml+1m_l=W_{l+1}^{\top}m_{l+1} (Eq. (16), with no activation derivatives along the way), and combines it with the free-equilibrium activities into a three-factor local update (Eqs. (17)(18)); Appendix A states that NEL is not equivalent to the nudged dynamics of EP. On MLPs for MNIST/FMNIST and a two-layer convolutional network for CIFAR-10, over six random seeds, NEL is compared with three-phase REP and three-phase EP, with two further controls, DCF (the sparse reward replaced by the full supervised error t−y0t-y^{0}) and truncated BPTT (T2=12T_2=12): NEL approaches three-phase REP in the shallow settings, and the gap widens as the setting changes; width, activation, T1T_1, epochs and learning rates change along with it, and the CIFAR-10 setting also changes both the dataset and the architecture type. DCF shows no systematic advantage over NEL and remains below three-phase EP in every setting; BPTT is close to or slightly above three-phase EP in the three MLP settings and remains about 1.9 percentage points lower on CIFAR-10. The paper also reports ablations on the exploration probability and on the free-relaxation length, and the layer-wise alignment analysis of Appendix B.1.

Critical strengths
  1. The authors state that NEL retains only the free phase, eliminates the reward-nudged equilibrium and the contrastive update, and updates through a three-factor local rule that involves no activation derivatives (pp. 2, 5; "retains only the free equilibrium" on p. 9). They position NEL as a single-equilibrium framework and list sparse reward-based teaching, reciprocal feedback, and derivative-free local plasticity as its distinguishing features (p. 9). On the authors' account, NEL retains the free-equilibrium computation while eliminating the subsequent reward-nudged relaxation and contrastive comparison (p. 5), and it keeps per-connection local updates, making "single equilibrium" an implementable architectural choice.
Critical weaknesses
  1. The experiments do not separate "retaining the free phase" from "a single feedforward sweep". The free phase enters Eqs. (15), (17) and (18) only through h0h^{0} (including y0y^{0}), and Eq. (16) contains only mm and W⊤W^{\top}. The paper has neither a γ=0\gamma=0 control nor a single-feedforward-sweep control; Figure 2b reports how the length of T1T_1 affects maximum test accuracy, but does not measure the quality of h0h^{0} and does not distinguish equilibrium relaxation from a single feedforward sweep.
  2. The CIFAR-10 row changes T1T_1, weight decay and the learning rate together, and the optimizer is not stated, so the NEL–REP gap cannot be attributed to the number of phases alone; the 6.21 in the abstract is the NEL–REP difference in this row.
Questions for authors
  • The nudged phase of EP implements the BPTT gradient, whose computation coincides step for step with the forward-time dynamics (Ernoult et al., 2019); what would Table 1 become if the reciprocal feedback were set to zero (in which case the fixed point of Eq. (23) is the ordinary feedforward activation), or if a single feedforward sweep were used instead with everything else unchanged? If the readings were essentially unchanged, how would the authors define the role of free relaxation (relative to a single feedforward sweep) in NEL?
  • Could the authors add a conceptual figure or an algorithm box that maps Eqs. (3)(6)–(10)(15)–(18)(23)(24) onto the computational procedure?
  • How does this explicit, layer-ordered modulatory pathway, which involves no activation derivatives and uses transposed forward weights, differ from random/direct feedback alignment (Lillicrap et al., 2016; Nøkland, 2016)? Could BrainProp (Pozzi et al., 2020), closely related to this modulatory pathway, also be included as a baseline?
  • On CIFAR-10, could the authors provide reruns with NEL@T1=120T_1=120 or the baselines@T1=80T_1=80, reruns with weight decay aligned, and reruns with the optimizer and learning rates aligned? Did NEL/DCF use weight decay in that row?
  • What are the values of ε\varepsilon in the main experiments, the optimizer and free-relaxation initial values for NEL/DCF, the nudging factor β\beta, and Δt\Delta t for BPTT/REP/EP in Table 2?
  • The last sentence of the abstract states "computations introduced by additional nudged relaxations become increasingly important", whereas §3.1 and the Discussion state "cannot be attributed solely …" / "rather than simply on the number of equilibrium phases"; how do the authors reconcile the two statements? How do the authors explain BPTT exceeding EP on the two-hidden-layer FMNIST (+0.34+0.34)?
  • Compared with three-phase EP, what NEL saves is the number of phases, but what physical quantity carries the additional explicit layer-by-layer transposed pathway on an analog substrate? Does this change EP's premise that updates need only quantities locally available at the synapse (Scellier & Bengio, 2017) and, with it, the hardware implications of that premise?
Reviewer_Nh2WOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper the authors propose Neuromodulated Equilibrium Learning (NEL), a single-equilibrium reward-based framework inspired by Attention-Gated Brain Propagation and neuromodulatory signaling. NEL does not require reward-nudged equilibrium and contrastive updates, as it propagates a reward-dependent modulatory signal derived from the selected action. In shallow architectures, NEL performs comparably to prior methods like three-phase REP, however, in deeper architecture the gap in performance becomes more noticeable.

Critical strengths

The motivation of the paper is strong and is highly relevant in the current era of ML. While backprop has emerged as the goto training framework, it is fundamentally bio-implausible and lacks the operational simplicity of STDP like bio-plausible learning rules. So, I believe its a highly relevant research direction to develop alternate learning frameworks for neural architectures.

Critical weaknesses
  1. The evaluation done for the work can be more thorough. Fashion MNIST and CIFAR 10 are both really simple datasets at this points and the method fails to achieve comparable results to its baseline on both.
  2. Following up on point 1, I believe scalability of this approach will be challenging. While EP itself is difficult to scale, the proposed single equilibrium based NEL might be even more so.
  3. Beyond Table 1 (where NEL does not even outperform in terms of performance), there is no convincing result/evidence that will motivate a researcher to use NEL over the other baselines.
Questions for authors
  1. Can the authors provide results on bigger datasets like CIFAR 100, Imagenet, etc?
  2. What is the primary challenge of scaling to this datasets? Is it poor performance, training cost, etc.
  3. How does the proposed method NEL, compare in terms of operational cost w.r.t other baselines. What is the motivation of using NEL and not any other EP approach? Please provide quantitive results for them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.