acceptodds
Under review as a conference paper at ICLR 2027

Local Credit Assignment under Dale Constraints with Paired Non-Negative Channels

Abstract

Biologically plausible credit-assignment models seek to explain how neural circuits can learn effectively under the constraints of real neurons. Despite significant progress, existing biologically motivated credit-assignment models generally do not jointly accommodate non-negative neuronal activity, fixed excitatory/inhibitory identity, and local synaptic learning. Under these constraints, neurons cannot directly represent negative activity or change the sign of their outgoing influence, making the propagation of signed learning signals particularly challenging. We introduce a biologically motivated architecture that represents signed quantities through paired non-negative channels, with sign encoded by channel identity rather than negative neuronal activity. A simple excitatory-inhibitory on-off motif implements this representation and is repeated throughout bottom-up and top-down pathways. Combined with a local activity-based learning rule, paired top-down channels provide opposing contributions to synaptic updates, yielding an effective signed credit-assignment signal using only local interactions and fixed-sign circuitry. We show theoretically that, when bottom-up and top-down connectivity are aligned, the resulting updates exactly recover backpropagation despite propagating only non-negative neural signals. Empirically, the model learns effectively across image-classification benchmarks, remains effective with non-aligned bottom-up and top-down pathways, and scales to Tiny ImageNet, showing that the proposed biological constraints need not come at the cost of effective representation or learning. These results provide a concrete mechanism for local credit assignment under non-negative activity and Dale-constrained connectivity.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Biologically Plausible Learning, Credit Assignment, Dale-Constrained Neural Networks, Local Learning, Hebbian Learning, Excitatory-Inhibitory Networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TtazOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors introduce an on-off circuit motif for "biologically plausible" credit assignment, which accomodates nonnegative firing rates, Dale's law, and layer-local learning. For the base models they consider, they get comparable performance on MNIST, Fashion MNIST, CIFAR-10, and Tiny ImageNet.

Critical strengths

For the base models the authors consider, they get comparable performance on MNIST, Fashion MNIST, CIFAR-10, and Tiny ImageNet.

Critical weaknesses
  • Dale's law is not necessarily biologically plausible, so it is unclear to me why this is a principal motivation for the authors (and seems awfully restrictive on what the learning algorithm can do, but see also my next point below)

  • The base models used are not great and are quite below SOTA top-1 performance on these tasks, even with the backprop baseline in Tables 1-2. For example, with modern CNNs, SOTA performance on MNIST is 99.9% (not 97-98% as in Table 1), SOTA performance on Fashion MNIST is 99.65% (not 87-88% as in Table 1), SOTA performance on CIFAR-10 is around 99% (not 54-56% as in Table 1), and finally SOTA performance on TinyImageNet is 66-67% not 42-43% as in Table 2.

Questions for authors
  • Related to the base model weakness, how many layers are the networks used?

  • Can you try ImageNet-1k with 224x224 images (rather than 64x64 images), and compare with other methods like Information Alignment (IA) of Kunin et al. ICML 2020 (https://arxiv.org/abs/2003.01513) which maintains positive firing rates via relu and scales to backprop with hundreds of layers on ImageNet?

Reviewer_SXkWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to explore a more biologically plausible neural update rules that complies (1) non-negative neuronal activity and (2) fixed excitatory/inhibitory neuron identity, while keeping (3) local synaptic learning. The authors show this possibility by transforming the architecture with paired non-negative channels and the sign is fixed. By construction, when the bottom-up and top-down connectivity are aligned, then the update rule is the same as backpropagation, which also means that this paper provides an interesting reformulation of backpropgation.

Critical strengths
  1. This work provides a formulation in which the backpropagation can be reached with non-negative activity and fixed excitatory/inhibitory neuron identities. It can be seem as a trivial reformulation, but the reformulation itself might be the most interesting part. Based on the understanding, the BU pathways carry the on-off activations and the TD pathway carries the ON-OFF error information. The update rule is local, which in the authors' term, thus the learning is local.
  2. The authors provide extensive experiments to validate their claims
Critical weaknesses
  1. The general work is a reformulation of BP using signed neurons, CReLU, and proposes to approach weight transport by construction. However, each part had been discussed pretty thoroughly in the previous literatures. For example, it seems to be adding the CReLU component from the [Deep learning without weight transport] paper, making it a more trivial addition to make the framework more biological plausible.

  2. The use of "Hebbian learning" term is also misleading. The update rule proposed by the author is not Hebbian, which is mostly unsupervised learning algorithm with only-local information (there is no error backpropgation, no matter how the error is built or represented) using the presynaptic and postsynaptic information. Can the author explain their definition of the "Hebbian learning"?

  3. Empirical evidence-wise, I found a few questions that would require the authors' rebuttal or clarifications. Some of the detailed set up and implementation are lacking, violating the train-val-test split, or violating the biological plausibility. Details are listed in the next section.

Questions for authors
  1. In L958, the authors mention "Results are reported as test accuracy at the best epoch during training, averaged over 10 random seeds" There is no validation set to pick the hyperparameters and is more like hacking the evaluation matrices. Can the author provide a seperate evaluation with standard train-val-test split?
  2. As On-Off BP is the same as On-Off Aligned (theoretically), why are their results in Table 1 different?
  3. The parameter selection is unclear to me: how is the CIFAR-selected being used? Is it selected for the On-Off model only? or for different setup (like for BP), the selected configs are different?
  4. In the "Learned Unit" setting, how is each motif weighted being updated? is it using BP? If they are learnable, do they stay in the same sign? Also, the shared learned unit setting is a violation of local learning.
  5. In Table 1, how is the vanilla Conv being trained? What is the difference between Vanilla Conv and CReLU?
Reviewer_R5R8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper represents every signed scalar by a pair of non-negative "on/off" channels, y±=ReLU(±(x1−x2))y^\pm=\mathrm{ReLU}(\pm(x_1-x_2)), produced by a small excitatory–inhibitory motif. All inter-layer weights are excitatory, with magnitude kept non-negative via ∣θ∣|\theta|. A top-down (TD) stream, with two non-negative populations per channel and BU-gated feedback, carries the error. Each weight is updated with a three-factor "Hebbian" rule, ΔWij=ηxj(ei+−ei−)\Delta W_{ij}=\eta x_j(e^+_i-e^-_i). A theorem shows that with TD weights equal to the transpose of the BU weights, the update equals backprop. Matched BU/TD updates keep alignment if present at initialization, and increase relative alignment from independent initialization. Experiments:

  • 2-layer MLPs on MNIST/FMNIST/CIFAR-10, roughly matching BP.
  • A 5-layer CNN on Tiny ImageNet, with aligned feedback only: 42.3% top-1, vs. 37.1% for a width-doubled ReLU CNN and 43.9% for CReLU trained with BP.
Critical strengths
  1. A clean, constructive circuit that satisfies three constraints at once: non-negative activity, fixed-sign synapses and a local update. The exact-BP theorem is correct and simple, and the paired-channel representation is easy to analyze. This is a useful reference construction for the bio-plausible learning community.
Critical weaknesses
  1. The constraints do not restrict the model, so matching backprop holds by construction. With the default fixed motif, a unit outputs ReLU(±(x1−x2))\mathrm{ReLU}(\pm(x_1-x_2)), where x1−x2=(∣W1∣−∣W2∣)h+(b1−b2)x_1-x_2=(|W_1|-|W_2|)h+(b_1-b_2). Because ∣W1∣−∣W2∣|W_1|-|W_2| can take any real value, each layer is an unconstrained signed linear map on the concatenated [h+;h−][h^+;h^-]. That is exactly a CReLU network (Shang et al., 2016), reparameterized with two non-negative matrices. The "Dale constraint" therefore limits the parameterization, not the function class. The non-negative splitting of signed weights is a known device (e.g., Parisien et al., 2008).

    With TD equal to BU transposed, Theorem 1 then says the rule is backprop on this network, which Table 1 confirms (On-Off BP 56.43 vs. Aligned 56.48 on CIFAR-10). The claim that "biological constraints need not come at the cost of learning" is therefore an identity, not an empirical finding.

    The supplementary code also does not fully enforce non-negativity. The motif inputs x1,x2x_1, x_2 are linear sums with signed, unconstrained biases and are never rectified (F.linear(x, |W|, b) feeds on_off_activation directly), and the network inputs are mean-normalized pixels. So the "non-negative input neurons" of Fig. 1 can carry negative activity.

  2. The scaling result relies on weight transport, and the transport-free regime is the known Kolen–Pollack rule, tested only at small scale.

    • Tiny ImageNet is run only in the aligned setting. The supplied config sets shared_weights: true (TD reads the BU weights directly) and uses AdamW. The 42.3% result is therefore BP+AdamW on a CReLU-type CNN. It says nothing about whether local credit assignment scales.
    • The baseline comparisons are not like-for-like. SCFF/DFA/Hebbian methods use no weight transport, and CReLU-BP is better still (43.9%). The BP-trained vanilla CNN (35.6%) is no better than SCFF (35.7%), which suggests the BP baselines were not tuned; the only Tiny ImageNet config is the on-off one.
    • The independent-initialization regime is Kolen–Pollack. Identical updates to independently initialized forward and feedback weights (Eqs. 36–39) are essentially the Kolen–Pollack rule (Kolen & Pollack, 1994). Akrout et al. (2019), which the paper cites without making the connection, already showed it trains ResNets on ImageNet.
    • Here that regime is tested only on 2-layer MLPs, and it needs a CIFAR-specific 0.1× initialization, 4× learning rate and 2× epochs. Under the common hyperparameters it falls to 41.7% on CIFAR-10.
    • MLP accuracies are best-epoch test accuracy with no validation split (main.py), while most gaps in Table 1 are ≤0.3 points.
Questions for authors
  1. (Could change my assessment.) Please run Tiny ImageNet, and a deeper CIFAR CNN, with separately parameterized, independently initialized feedback and plain SGD. Compare against Kolen–Pollack and feedback alignment on the same on-off architecture.
  2. (Could change my assessment.) Since x1−x2x_1-x_2 spans all signed weights, what does the Dale constraint actually rule out here? Is there any function or learning trajectory a CReLU network can realize that the on-off network cannot?
  3. (Clarification.) Do the results hold if x1,x2x_1, x_2 are rectified, the inputs are non-negative, and biases are carried by excitatory/inhibitory units rather than signed scalars?
  4. (Clarification.) Please report MLP results with model selection on a validation split, and tuned BP baselines on Tiny ImageNet.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.