acceptodds
Under review as a conference paper at ICLR 2027

Reassessing the Biological Plausiblity of Backpropagation

Abstract

Backpropagation is often called into question on the grounds of biological implausibility and incompatibility with analog and neuromorphic hardware. Alternatives to backpropagation range from heuristic modifications to principled energy-based contrastive learning algorithms. However, the criteria for what constitutes biological plausibility vary and derive from a reductive definition of BP, which conflates the choice of algorithm and network topology, exclusively considering BP in the feedforward/DAG setting. In this work, we revisit the method of Lagrange multipliers, which, when applied to credit assignment in a neural network, yields recurrent BP with standard BP and BPTT as special cases. This frequently overlooked Lagrangian formulation of BP, previously explored in LeCun (1988) possesses many of the merits and locality properties favored by the literature on local and bio-inspired learning. We highlight that recurrent BP dynamics are fully local and that a common objection, the distinction between activity states and error states, can be bypassed with a simple change of coordinates. We demonstrate experimentally on networks with asymmetric topology that the gradient estimates obtained in both coordinate systems are equivalent, with parameter-matched models out-performing standard feedforward backpropagation in the asymmetric setting. We additionally derive a variant of dyadic RBP which may be more hardware friendly at the cost of introducing an approximation error term. These results suggest that backprop is substantially closer to biological plausibility than is typically assumed, and may be more compatible with analog and neuromorphic computing paradigms than previously believed.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
bioplausibility, backpropagation, local learning, biological plausibility, dyadic learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JwhDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper revisits the biological plausibility of backpropagation through the constrained-optimization formulation of recurrent backpropagation (RBP). It introduces dyadic coordinates that encode network states and adjoint signals as weighted averages and differences of two state variables. The authors establish exact equivalence between the original and transformed dynamics, and propose a split surrogate with a second-order approximation error under smoothness assumptions. Experiments examine gradient consistency, CIFAR-10 classification, and image denoising under different connectivity structures.

Critical strengths
  1. The paper clearly distinguishes gradient computation from the conventional scheduling of forward and backward passes. Representing adjoint signals through differences between paired states provides an informative perspective on the dedicated-error-channel objection to backpropagation.
  2. The paper connects standard BP, RBP, and dyadic formulations through a common constrained-optimization framework. By showing how predictions and adjoint signals can be represented as weighted averages and differences of paired states, it clarifies the mathematical relationships between these methods and distinguishes exact reparameterization from approximate splitting.
Critical weaknesses
  1. The incremental contribution over closely related work needs sharper justification. The paper acknowledges that the symmetric dyadic formulation coincides with dyadic equilibrium propagation, and that related splitting procedures already exist. Generalizing the mixing coefficient, constraints, and split parameter updates is meaningful, but the experiments primarily confirm gradient equivalence and comparable task performance. They provide limited evidence that these extensions enable capabilities or implementation benefits unavailable to the closest predecessors.
  2. The biological interpretation requires a clearer connection to implementation. The coordinate transformation establishes that adjoint information can be encoded through differences between paired states, but it does not establish how those states and their coupled dynamics could be realized by neural or synaptic mechanisms. The unresolved Jacobian-transpose requirement is acknowledged by the authors, yet the claimed comparability to other biologically motivated methods would benefit from an explicit comparison of the required computations and circuit assumptions.
  3. Limited empirical support for the proposed implementation benefits. The theoretical analysis and gradient comparisons adequately support the distinction between exact reparameterization and approximate splitting. However, the suggested hardware advantages remain largely prospective. Although relaxation iteration counts are reported, the paper does not directly evaluate whether the split formulation reduces computational cost, memory usage, or implementation complexity relative to RBP and the non-split dyadic formulation.
Questions for authors
  1. The paired-state representation offers an informative perspective on error encoding. How could these states and their coupled dynamics be implemented biologically, and which implementation constraints does this formulation alleviate given the remaining Jacobian-transpose requirement?
  2. The unified framework usefully clarifies the relationships between BP, RBP, and dyadic methods. What capabilities or implementation benefits do the proposed extensions add beyond the closest predecessors, and can controlled comparisons demonstrate gains in computational cost, memory usage, or convergence?
2 comments
AuthorsOpenReview ↗

Response to reviewer JwhD

We thank the reviewer for their comments which encouraged us to sharpen several points in the motivating section of the paper.

W1

When simulated on GPUs RBP, dyadic RBP (with α=0.5\alpha=0.5 e.g. "dyadic EP"), and split dyadic RBP perform comparably. Changing the topology (skew-sym, sym, asym and feedforward) leads to noticeable differences in performance. The split formulation has the benefit of not requiring evaluating constraints or derivatives at the midpoint, which prior works have highlighted as a potential limitation for neuromorphic hardware. The cost is that the split formulation introduces a β\beta dependent approximation error which we show can be circumvented by setting alpha to 1 or 0. Splitting techniques applied in prior works have either been limited to one specific potential (Scurria et al 2026) or have been presented without derivation (Scurria 2026). The alpha reparametrization offers us a way to avoid evaluating constraints and derivatives at the midpoint without approximation error –a genuine strength of our algorithm– since the edge cases α∈{0,1}\alpha\in\{0,1\} give equivalent formulations. Although a performance difference in terms of accuracy is not noticeable we believe the α=1\alpha=1 and α=0\alpha=0 setting are interesting from a neuromorphic computing perspective because the circuit is comparatively simpler than the centered case explored in prior works. It should therefore in principle be more amenable to implementation on physical substrates, where significant benefits in energy efficiency and/or inference/training speed have been theorized, and where the requirement of infinitesimal nudging from previous best prospects (Eqprop) are a barrier in scaling on noisy devices. This is a novelty of our work that may have utility for analog computing and we demonstrate it in our main classification results (final row) and appendix F. We provide more details on our contribution and how we build on prior works in our answer to Reviewer hXnx’s question Q2

W2

This touches on the main message of our paper (See line 53-55 in the introduction and line 479-481) in the conclusion. The motivation for this work was that we noticed that the term “biological plausibility” is used very liberally in the EP and PC literature and that by definitions established in those works BP and RBP are just as plausible as EP and PC. The implication being that the bar for calling something a biologically plausible alternative to backpropagation should be set higher than is typically done in the literature. We propose to introduce a table summarizing the commonly used criteria and indicating which algorithms check which boxes. You find this table in our general rebuttal to all reviewers above.

W3

The benefit of the split formulation is that it avoids the need to evaluate constraints and derivatives at the mid-point. Arguably, the settings$ \alpha=1$ and α=0\alpha=0 where the dyadic RBP and split dyadic RBP formulations are equivalent are the most appealing from a hardware perspective as only one of the two states receive an error perturbation, and as the beta dependence vanishes (weak nudging is problematic in noisy analog hardware). Since submitting the paper we have completed additional experiments with ResNet18 type asymmetric networks achieving SOTA performance in the local learning algorithm literature on CIFAR10 and ImageNet32. For details about runtime and memory requirements please see our answer to Reviewer ihBj’s question Q2.

Q1

The main difference is what type of information a neuron stores. In RBP we store both the activity and the error, whereas in dyadic RBP and split dyadic RBP we store two oppositely perturbed states. Previous work has suggested that such a thing could be accomplished via the integrative function of a cell-body in relation to its apical and basal (dyadic) dendrites. The dual/integrative nature of neurons is a fact of biology. However any effort to tie our formulation precisely to such biology, we believe, would be overly speculative. In the case of symmetric and skew-symmetric connectivity (split) dyadic, RBP can be implemented in two distinct phases with single compartment neurons –resulting in a circuit that only propagates activity (and not errors), though this is hardly “bioplausible.” Please see our rebuttal to ihBj Q1.

Q2

We believe we have addressed this in our previous responses. Please take a look at our responses to reviewer ihBJ, especially Q2, regarding computational cost and memory. As we mention in our general rebuttal, we propose to add a table comparing runtimes, and a general discussion of this matter in the paper body.

AuthorsOpenReview ↗

New tables

We would just like to draw reviewer's attention to the new computational cost benchmarks and SOTA results added in general rebuttal.

Reviewer_ihBjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper revisits the biological plausibility of backpropagation through the Lagrangian formulation of recurrent backpropagation (RBP). The authors propose Dyadic RBP and Split Dyadic RBP, which reformulate the original dynamics using coupled neural states. Theoretical analysis establishes exact equivalence for Dyadic RBP and bounded approximation errors for Split Dyadic RBP. Experiments on image classification and denoising with asymmetric recurrent networks validate the gradient estimates and demonstrate competitive performance.

Critical strengths
  • The paper provides a useful reconsideration of backpropagation by distinguishing its underlying credit-assignment mechanism from conventional feedforward implementations.
  • The proposed methods are supported by mathematical derivations establishing exact or approximate equivalence to recurrent backpropagation.
  • The experiments extend the investigation beyond conventional feedforward and symmetric networks, demonstrating the applicability of the formulations to asymmetric recurrent architectures.
Critical weaknesses
  • The proposed methods primarily reparameterize existing RBP dynamics without demonstrating that the resulting computations can be implemented through biologically plausible neural mechanisms. Mathematical equivalence alone does not establish improved bioplausibility.
  • The proposed methods still require transposed-Jacobian information during training. This fundamental biological limitation significantly weakens the paper's central motivation.
  • The evaluation is limited to relatively small networks and datasets, with no direct comparison against closely related dyadic learning methods. Moreover, the modest accuracy improvements do not establish practical advantages, particularly without training-time, memory, or computational-cost comparisons.
  • Dyadic RBP with α=1/2\alpha=1/2 coincides with existing Dyadic Equilibrium Propagation, while related coordinate transformations have already been explored. The novelty and practical significance of the proposed generalizations are insufficiently distinguished from prior work.
Questions for authors
  • Beyond a mathematically equivalent coordinate transformation, what concrete biological constraints are eliminated by Dyadic RBP? Can the authors specify a plausible circuit-level implementation that does not implicitly reconstruct the original adjoint error computations?
  • How do the methods compare with conventional BP, RBP, and related biologically inspired algorithms in terms of actual training time, memory use, and convergence reliability?
  • Since the proposed algorithms still require transposed-Jacobian information during training, how would they behave when exact weight transport is replaced with biologically motivated approximate feedback? Would the resulting gradients remain sufficiently accurate?
4 comments
AuthorsOpenReview ↗

Response to reviewer ihBj

We thank reviewer ihBjfor their thoughtful commentary.
W1
Please see line 53-55 and line 479-481. In the BP-alternative literature, “bioplausibility” often encompasses various criteria which we summarize in the paragraph starting on line 069. Our goal was to show that in dyadic coordinates RBP is as good or better than the most promising proposed local learning algorithms. To make this more clear, we propose a table comparing approaches across these specific criteria.
W2
Weight transport is indeed a limitation for bioplausibility. This limitation is not specific to RBP though. EP, CHL, DualProp and PC variants that fully remove weight transport are heuristic modifications of their parent algorithms and will inevitably compromise gradient quality. The same heuristic modifications can of course also be applied to RBP, dyadic RBP and split dyadic RBP (e.g. Kolen-Pollack learning, weight mirrors or feedback alignment). Regarding the paper’s central motivation please see our response to Q1.
W3
Dyadic EP is just dyadic RBP with α=1/2\alpha=1/2. So it is covered in table 1. We did not reimplement dyadic learning or dual propagation, but expect they would perform equally well as long as a sufficiently small β\beta is used. Regarding the scale of the experiments we have implemented our algorithm on an asymmetric ResNet-18, achieving SOTA accuracy (in the local learning algorithm niche) on imagenet32 and Cifar10. It is worth noting that the CNN results in Høier et al 2023, Høier & Zachl 2024,Scurria et al 2026 and Scurria 2026 were all restricted to feedforward networks.

You are correct that simulating the relaxation of the dyadic potentials on a GPU will be slower than ff on GPU. For asymmetric nets to be competitive we would need to identify hardware capable of efficient relaxation. In the symmetric formulation of our Split Dyadic RBP (especially with alpha=1/0--a novel contribution of this work), we get similar analog-friendly properties of e.g. EP around which a large body of literature has developed projecting significant gains in energy efficiency and inference speed.
W4
The short answer is: We apply the alpha-weighted reparametrization of Høier & Zach 2024 to a much more general class of networks.This allows us to avoid the approximation error of EP-like algorithms while maintaining its amenability to local hardware and “bioplausibility.” While our formulation recovers Dyadic EP as a special case, we show that dyadic "EP" is actually RBP. We also show (or remind) that BP and RBP are not inherently non-local and dispel the notion that LeCun’s formulation leads to unstable dynamics (a claim made both in Seely et al 2026 and Scurria 2026). Finally we provide the most comprehensive study of “split” dyadic parametrizations to date, and achieve SOTA (within the local learning niche) on multiple datasets. See our response to hXnx’s question Q2 for further details on our contribution and prior work.
Q1

Our claim is not that we have found a new unconditionally bioplausible learning algorithm, but that by the standards of locality and bio-plausibility employed in the EP and PC literature RBP is just as "bioplausible" as EP and PC. Please see our general response and new table. Regarding explicit error computations, there are two cases where the need to propagate errors vanishes: (1) if the network is symmetrically connected and (2) if it is skew-symmetrically connected. In both cases the problem disentangles into two sequential phases in which only activity is propagated. We refer to Høier et al 2024 for more information on the second case.
Q2

The difference in runtime is a question of topology. FF networks which can be fully relaxed in a single traversal take about the same time whether we use dyadic RBP, RBP or BP with autograd. All recurrent settings need many relaxation timesteps, hence they are much slower to simulate than FF nets. Autograd(BPTT) is not practical for our recurrent models as the memory cost scales with the number of timesteps. RBP on the other hand only needs to store the final equilibrium’s ss and δ\delta. If we disregard the size of the weights then BPTT will require num_timesteps/2 times as much memory as RBP, dyadic RBP and split dyadic RBP. To clarify this we propose adding a table of runtime comparisons and a statement on memory burden. Convergence is guaranteed in symmetric and feedforward topologies; skew- and a-symmetric require sufficiently small step-sizes. All of these details are discussed in the appendix.

Q3

We expect the same behavior as is generally observed in FA literature. For shallow dense networks learning is expected to proceed effectively and for deeper networks and for convolutional networks in particular sufficient gradient alignment will require mechanisms encouraging the error channel’s weights to match the transpose of the activity channel’s weights (e.g. Kolen-Pollack learning or weight mirrors).

AuthorsOpenReview ↗

New tables

We would just like to direct the reviewer's attention to new results on computation cost and SOTA results added in general rebuttal.

I think the authors have addressed all the questions I mentioned.

Thank you again for your constructive commentary, criticism, and for increasing your score. We appreciate your support of this work!

Reviewer_hXnxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper revisits Lagrangian formulation of recurrent backpropagation (RBP), where standard BP is the special case. Applying the dyadic change of coordinates of [Høier and Zach (2024)] with a mixing variable α∈[0,1], it derives dyadic RBP (which is equivalent to RBP) and then split dyadic RBP, in which the weight gradients match RBP up to O(α(1-α)β²). Experiments on asymmetric CNNs show that both variants match RBP as a sanity check.

Critical strengths
  1. The major contribution of this paper is to restructure the Lagrangian of RBP with the dyadic reparameterization while allowing asymmetric mixing of neuron states, obtaining new algorithms.
  2. The theorem and the proof is clearly written.
Critical weaknesses

Overall, I think this paper has limited novelty and contribution, for the following reasons.

  1. The dyadic change of coordinates applied to the BP Lagrangian was proposed by [Høier (2024)]. Although Section 1.2 acknowledges this, Section 3 opens with "We introduce a change of variable…" (L178) without a citation, which can be read as a new contribution. Please cite Høier (2024) there.

  2. The mixing factor α is also not new. Given Eq. 7, Eqs. 8–10 follow mechanically.The exactness in Theorem 3.1 is then immediate, since an invertible linear change of variables leaves the trajectories of an ODE unchanged.

  3. The bio-plausibility properties claimed for dyadic RBP are largely inherited from prior work. Locality is already a property of RBP dynamics (as claimed by Lecun), and encoding errors as differences of dyadic states, which avoids a separate error channel, is the premise of dual propagation.

  4. The α-dependence in Theorem 3.3 is less informative than presented. For fixed α∈(0,1), O(α(1−α)β²) = O(β²), the same order as the α=1/2 splitting in Scurria et al. (2026). The prefactor α(1−α) is valid, but it is simply the variance of a two-point distribution, not an error analysis.

Questions for authors
  1. Theorem 3.3 is expressed at fixed points. Can the results be different at the equilibrium points of the dynamics?
  2. Can the authors clarify the main results of Hoier 2024 and Scurria 2026 and how your work differ from theirs?
4 comments
AuthorsOpenReview ↗

Response to reviewer hXnx

We would like to first thank reviewer hXnx for their helpful commentary, particularly for drawing attention to several places where our work might be more clearly distinguished from previous research.

Criticisms:
W1
It was not our intention to imply that the change of variable was first introduced to the literature here. We will repeat the citation here and change “we introduce” to ”we apply”.
W2
We agree that the math here is simple. The mixing factor is also present in Høier & Zach 2024. We address the novelty of our contribution in our answer to your question Q2.
W3
Please refer to our comment to Q2. We additionally refer to our general response where we include a table that clarifies how our work relates to other approaches in terms of the canonical “bioplausibility” criteria of this literature. See also line 53-55 in the introduction and line 479-481 in the conclusion, where we have laid out our motivation most directly.
W4
Could you clarify if there is a specific claim or statement you suggest we should temper?

Q1
We are not sure what you mean here. Could you clarify?

Q2
Lecun 1988 considered a specific constraint function for feedforward networks and for recurrent networks and only worked in the classical activity/error coordinates.
Høier & Zach 2024 (adjoint dualprop), introduced a α\alpha-weighted dyadic parametrization of a specific Lagrangian associated with a feedforward network. This was a follow-up paper to the original dual propagation paper (Høier et al 2023).
Høier et al 2024 (Dyadic Learning) derived learning rules for networks with asymmetric feedback by way of the optimal value reformulation, but focused experiments specifically on the skew-symmetric, symmetric and feedforward settings. As LeCun’s RBP and adjoint dualprop assumes a specific form of the constraints, so does dyadic learning restrict analysis to a specific energy function.
Scurria et al 2026 (EP for nonconservative systems) also applies to recurrent networks but unlike dyadic learning and dual propagation does not make restrictive assumptions on the nature of the underlying constraints. Unlike adjoint dualprop, Dyadic EP only considers the α=½\alpha=½ setting. A companion paper Scurria 2024 (A Physical Theory of Backpropagation…) applies dyadic EP to feedforward networks and contrasts their approach to LeCun’s Lagrangian formulation of backpropagation, claiming that LeCun’s dynamics will not converge (this misconception is also found in Seely et al 2026). We show that this is not the case.
Our contribution We apply the alpha-weighted dyadic parametrization of Høier et al 2024, but like Scurria et al 2026 we do not restrict ourselves to a specific constraint function. While we are combining insights from prior works we believe our insights and analysis are novel. Firstly our algorithm is much more general. The alpha-parametrized dyadic RBP variant subsumes dyadic EP as a special case and shows that it is equivalent to LeCun’s RBP formulation. This both dispels some perceived shortcomings of LeCun’s Lagrangian backpropagation and shows that dyadic “EP” is best understood as RBP in a particular coordinate system. Secondly, we provide a derivation of the split variant and show that with α=1\alpha=1 and α=0\alpha=0 the dyadic RBP and split dyadic RBP formulations are equivalent. Splitting techniques applied in prior works have either been limited to a specific potential (Scurria et al 2026) or have been presented without derivation (Scurria 2026). Our split dyadic variant has potential benefits on analog devices which we discuss in our response to reviewer jwHd’s first comment (W1), and will further highlight in our proposed adjustments to the manuscript. Finally (see general comment) we have as of now achieved SOTA (in the local learning algorithms niche) on a deeper (ResNet-18) network for both CIFAR-10 and Imagenet-32. This should be compared to the previous dyadic approaches for which asymmetric networks have only been explored in the MLP setting, with CNN implementations restricted to feedforward connectivity.

As the discussion period comes to a close, we wanted to follow up to ask whether our rebuttals have addressed the reviewer's concerns. We wanted to also ask for clarification on points W4 and Q1 :

The α-dependence in Theorem 3.3 is less informative than presented. For fixed α∈(0,1), O(α(1−α)β²) = O(β²), the same order as the α=1/2 splitting in Scurria et al. (2026). The prefactor α(1−α) is valid, but it is simply the variance of a two-point distribution, not an error analysis.

and

Theorem 3.3 is expressed at fixed points. Can the results be different at the equilibrium points of the dynamics?

We were not sure what you meant and would like to have the opportunity to respond if possible.

We also would like to direct you to the general rebuttal where we have posted some new (SOTA) results , benchmarks for computational speed and a table clarifying how our algorithm compares to some extant BP-alternatives in terms of the various criteria of "bioplausibility" from the literature.

Again, we thank you very much for your helpful commentary and criticism, which we have found to be constructive.

The reviewer thanks the author for the rebuttal.

  1. The reviewer was first confused by having the alpha factor inside the big O notations. But it is not a very big deal.
  2. For Q1, I wonder if it is possible to study the stability of the equilibrium points, or it is not feasible.

Despite these points, I think the motivation is clear and pretty plausible. I had raised the scores.

We thank the reviewer for their comment and the increased assessment.

Regarding stability, yes, this should be feasible. Theorem 3.3 is at an arbitrary state (s,δ)(s,\delta) rather than at fixed points, so one could find local stability by linearizing dynamics there.

Again thank you for supporting our work and for all the constructive commentary.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Rebuttal

We thank the reviewers for their time and thoughtful commentary. In the reviews we have noted some recurring themes. Firstly, all reviewers requested clarification about where our algorithm falls in relation to prior research. Reviewers hXnx and JwhD specifically questioned the novelty of our approach. Reviewers JwhD and ihBj requested clarification regarding the bioplausibilty of our approach. Reviewers HXnx and ihBj requested experiments at larger scale as well as clarification regarding comparative computational advantages of our algorithm.

To respond briefly on what we discuss in greater detail below, we regret that the primary motivation and contribution of this paper was unclear. Our aim is not to present BP as a biologically plausible algorithm without qualification, but to demonstrate that via a coordinate transform backprop itself appears as bioplausible as any of the major local learning approaches, according to the specific criteria for “bioplausibility” presented in that literature. We are careful in our paper to acknowledge that this is not necessarily a reflection of actual biology, but can perhaps state this more strongly. In light of these criteria, our method –especially the split dyadic variant with α∈{1,0}\alpha\in\{1,0\} ( our most novel algorithmic contribution) has several favorable properties compared with previous literature. Additionally it is agnostic to connectivity, avoids approximation error, performs well in the strong nudging regime, and in certain cases can avoid weight transport (see ihBj Q1).

Pending the reviewers’ approval we propose to clarify these points in our paper via the following adjustments:

  • A small “contributions” section, detailing exactly what our work offers, in distinction to other prior, related research.
  • A table outlining runtimes of Dyadic, RBP and standard BP.
  • A small paragraph discussing the fact of similar computational burden of ours vs other algorithms, as well as the memory benefits of our approach.
  • A statement on the plausible advantages of implementation of our algorithm on alternative hardware.
  • A table (see below) summarizing where our algorithm falls among the canonical “bioplausibility” criteria presented in the beyond-backprop literature.

Knowing that our empirical contribution was limited, we ran additional experiments on a larger model (Resnet-18) with asymmetric connectivity, where we achieve definitive SOTA performance for all local learning literature on CIFAR-10 (94.7 validation acc) and Imagenet32 (46.0 validation acc). We also have promising results on full Imagenet which may be ready for inclusion by the end of the rebuttal period. This is a validation of the most interesting and novel variant of our work: split dyadic RBP with α=1\alpha=1. Therefore, alongside the above additions, we propose:

  • A small table showcasing our results in relation to previous best performance from local learning literature.
  • Appropriate edits to the abstract and discussion highlighting this empirical contribution.

Again we wish to thank the reviewers for their time and attention, and for their considerable help in enhancing the quality and clarity of this work.

Requirement / FeatureVanilla-BPEPPCDPRBP (Lagrangian)RBP (Dyadic)
Avoids weight transportPartlyPartlyPartlyPartlyPartlyPartly
Avoids dedicated error statesNoYesNoYesNoYes
Local learning ruleNoYesYesYesYesYes
Avoids update lockingNoPartlyPartlyPartlyPartlyPartly
Compatible with strong nudging–NoPartlyPartly–Yes
Exact gradient estimatesYesNoPartlyNoYesYes
Supports asymmetric connectivityNoPartlyYesNoYesYes

There are many nuances to each cell of this paper, which we intend to discuss in the table caption. Please see comments below and feel free to ask for clarification.

AuthorsOpenReview ↗

Additional Tables

We have run the proposed benchmarks for compute time both batchwise (bs=32; reported value is mean over 20 distinct batches x 4 seeds; for consistency on recurrent/asym we fix n-iterations at 15) and per-epoch.

Epoch-wise we we show runtime for fixed (15) iterations, as well as convergent (unlimited iterations, run until residual falls below a tolerance criterion). In practice some batches will require many more than 15 iterations and some many fewer, hence the overall reduction in runtime per convergent epoch compared to fixed iterations.

Note in general autograd is slower on recurrent networks, and only slightly faster on feedforward networks. Autograd is infeasible in convergent as it is full BPTT. e.g. stores every intermediate state of the unrolled free phase . This requires O(T) activation memory vs RBP and Dyadic where we recover gradients from the fixed point (O(1) in number of iterations).

As we mentioned elsewhere, (our) local variant is superior in terms of memory use. We now show our implementation is slightly worse on runtime in the feedfoward setting and slightly better in the recurrent setting than autograd.

MeasurementRBPDyadicSplitAutograd
One-batch, asym (ms/batch)24.80 ± 0.1125.68 ± 0.0625.12 ± 0.0632.41 ± 0.06
One-batch, ff (ms/batch)1.89 ± 0.001.95 ± 0.011.96 ± 0.011.19 ± 0.00
Epoch fixed-15, asym (s)81.4 ± 0.982.1 ± 0.482.2 ± 0.892.8 ± 0.6
Epoch fixed-15, ff (s)30.9 ± 0.132.2 ± 0.332.3 ± 0.324.7 ± 0.0
Epoch convergent, asym (s)69.8 ± 0.665.7 ± 0.367.6 ± 0.6—
Epoch convergent, ff (s)30.3 ± 0.131.8 ± 0.032.0 ± 0.024.7 ± 0.0

We also are adding the details of our SOTA / ResNet results Top-1 val acc across 3 seeds:

DatasetAsymFFPrevious best
CIFAR-1094.6 ± 0.194.6 ± 0.193.92 (Vinayak & Srinivasan, 2025)
ImageNet3246.5 ± 0.141.5 ± 0.146.0 (Nest & Ernoult, 2024)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.