acceptodds
Under review as a conference paper at ICLR 2027

PotentialEP: Backprop-Free Training of CIFAR-Scale Generative Models via Equilibrium Propagation

13 views

Abstract

Generating an image with a diffusion or flow-matching model typically involves integrating a velocity field for hundreds of sequential steps. That computation suits a GPU poorly and an analog neuromorphic substrate well, where one fast, low-power physical relaxation could replace the whole sequence. Two things are needed: a generative algorithm whose computation is a relaxation, and a way to train it without backpropagation, whose global gradient transport is hard to realise in analog. Energy Matching supplies the first, sampling by gradient-descent relaxation on a time-independent potential — but obtaining the gradient that descent follows is itself a backward pass, once per integration step. We prove that a predictive coding network with a single linear injection term added to its energy relaxes to a state whose visible layer carries that gradient inherently, with an error that vanishes for small injection strengths. An Energy Matching model trained by backpropagation can therefore be loaded into this network unchanged and sampled by relaxation alone, matching the original model's FID to within at CIFAR-10 and ImageNet-32 scale. The network can also be trained without backpropagation: Equilibrium Propagation reproduces backpropagation's parameter updates to a cosine similarity of , and training end to end with those updates reaches an FID of on unconditional CIFAR-10 against for backpropagation. To our knowledge this is the first CIFAR-10 generative model trained with Equilibrium Propagation, and the first at competitive quality by any backpropagation-free rule. This quality does not require group normalisation or attention: an architecture of only convolutions and pointwise nonlinearities, both native to an analog crossbar, reaches an FID of when trained by backpropagation.

Primary area:
generative models
Keywords:
equilibrium propagation, predictive coding, energy-based models, generative models, neuromorphic computing, local learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YTKDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Analog neuromorphic hardware can solve optimization problems that reduce to energy minimization with exceptional time and energy efficiency. Consequently, formulating generative models as energy minimization processes while rendering them backpropagation-free offers substantial efficiency benefits. This paper unifies Energy Matching, an energy-minimization-based generative framework, with predictive coding and equilibrium propagation. Specifically, it (i) computes the backprop-based potential gradient required for Energy Matching in a backprop-free manner via the prediction errors of a Predictive Coding Network (PCN), and (ii) eliminates backpropagation during training by adopting Equilibrium Propagation, which seamlessly aligns with PCN for local learning. In doing so, the authors replace the standard quadratic nudge with a linearized nudge to eliminate equilibrium displacement bias. Evaluation results show that the proposed model achieves high-fidelity generative modeling on CIFAR-10, matching the performance of backprop-trained baselines such as DDPM and Energy Matching.

Critical strengths
  • Addresses a timely and compelling problem by designing a fully backpropagation-free generative framework while clearly demonstrating a practical application scenario where such a model provides tangible benefits.
  • Successfully constructs the first generative model trained and sampled end-to-end via purely local dynamics, achieving performance on par with backpropagation-trained baselines.
Critical weaknesses
  • The technical novelty appears limited, as the paper mainly focuses on unifying three existing, correlated concepts into a single framework. While novel components such as the linearized nudge and linear injection terms are introduced, they feel highly specialized to this specific setup rather than offering broader conceptual insights to the community.
  • The paper is exceptionally dense and difficult to comprehend. The structure, phrasing, and overall prose strongly suggest heavy reliance on LLM generation at both the organizational and sentence levels. I strongly encourage the authors to revise the writing with human readers in mind to improve readability and flow.
Questions for authors
  • Related Work section needs significant reinforcement. Currently, it only provides a brief overview of predictive coding and energy-based generative models. A much more comprehensive survey is critical, specifically covering prior attempts to integrate predictive coding with equilibrium propagation, other local learning rules applied to generative modeling, backpropagation-free generative frameworks in general, and recent advances in predictive coding itself. Do the authors plan to expand this section in a revision, or do they believe the current positioning is sufficient? If the latter, please justify why.
  • The paper is exceptionally dense and severely hinders human comprehension. Was this intentional? If not, how do the authors plan to rewrite the paper to improve readability, clarity, and overall flow for the reader?
  • Prior literature has already explored combining predictive coding with equilibrium propagation [1], as well as applying equilibrium propagation to generative models [2]. Isn't this work essentially an incremental interpolation of these existing building blocks? Could you clearly summarize what conceptual breakthroughs set this work apart?
  • Is pixel-level generative modeling actually a primary or natural workload for neuromorphic substrates? For instance, biological brains do not perform dense pixel-level generation like diffusion models, but rather execute creative or generative tasks through action sequences and structured latent representations. Could you justify why hardware-constrained analog substrates are an appropriate platform for pixel-space image generation?

[1] Millidge, B., Song, Y., Salvatori, T., Lukasiewicz, T., & Bogacz, R. (2022). Backpropagation at the infinitesimal inference limit of energy-based models: Unifying predictive coding, equilibrium propagation, and contrastive hebbian learning. arXiv preprint arXiv:2206.02629. [2] Gower, A. (2026, August). FlowEqProp: Training Flow Matching Models with Gradient Equilibrium Propagation. In Proceedings of the International Conference on Neuromorphic Systems (pp. 60-66).

4 comments
AuthorsOpenReview ↗

Response to Reviewer YTKD

We thank the reviewer for the detailed review and for recognising that the paper addresses "a timely and compelling problem" with "a practical application scenario where such a model provides tangible benefits". The questions have substantially improved the revision, and we answer each below.

Q1. Related work. Yes, we have substantially expanded the "Related work" section, covering each area the reviewer named:

  • PC and EP together: Kerjan et al., who train PCNs by EP to ImageNet classification scale, and the PC–backpropagation equivalences of Whittington & Bogacz and Millidge et al., including the reviewer's reference [1].
  • Local learning rules beyond classification: EP-trained Hopfield VAEs, FlowEqProp [2], local rules on energies without hidden units (Bosch et al.), an EP-trained convergent Energy Transformer (Høier et al.), and Monte Carlo and Langevin predictive coding.
  • Backprop-free generative frameworks in general: we now state which generative models are actually backprop-free. Langevin PC's CIFAR-10 FID of 113 uses a decoder trained by backpropagation, and the physical, optical and spiking generators we survey are all trained by backpropagation.
  • Recent PC advances: PCNs trained at ImageNet scale (Kerjan et al.) and the error reparameterisation of Goemaere et al., which our solver builds on.

Q2. Density. The density was not intentional or due to extensive use of AI. It came largely from compressing a draft that was originally 13 pages long into 9 pages. We have used the tenth page to revise the prose and improve clarity throughout, and added a new Figure 1 that explains the PotentialPCN visually and a conclusion that summarises our contributions.

Q3. Novelty relative to [1] and [2]. Until now, generative models trained without backpropagation have either stayed at MNIST scale or fallen well short of competitive quality. We respectfully see reaching CIFAR-10 at backpropagation's quality as a conceptual step rather than an interpolation. The key idea is to make generation itself a relaxation on a PC energy: one equilibrium state then provides the sampling velocity, replaces the backward pass, and is what EP trains. This gives three advances.

  1. What the equilibrium carries. [1] shows that PC, EP and contrastive Hebbian learning approximate backpropagation at an infinitesimal perturbation of the output, for the parameter gradients of a supervised loss. Our injection term uses this mechanism in a setting [1] does not consider. There is no target, the quantity read is the gradient of the network's scalar output with respect to its input, and the PCN's own energy becomes γV\gamma V. The relaxation therefore is the generative sampler, and backprop-trained Energy Matching models load unchanged and match their FID to within 0.04 at CIFAR-10 and ImageNet-32 scale. Because the energy equals γV\gamma V, the contrastive phase also becomes a local contrastive Hebbian rule with no nudge at all.
  2. Making GradEP scale. [2] introduced GradEP on a Hopfield-type energy at MNIST scale. Two things take it to CIFAR-10. The PCN form lets GradEP train any feedforward architecture, here the same UNet+ViT as the backpropagation baseline. And we show analytically that GradEP's quadratic cost biases EP once β\beta approaches the tether stiffness, while finite precision needs large β\beta, and that the linearised nudge removes this bias, widening the usable β\beta from a single value to the range 3 to 10310^3.
  3. The result. The first EP-trained generative model at CIFAR-10 scale, with FID 3.33, on par with backpropagation's 3.54.

On generality: the injection applies to any scalar-output network, and Energy Matching unifies flow matching with energy-based modelling, so the construction applies wherever a generative model is expressed as a scalar potential. The linearised-nudge analysis applies to any use of GradEP with a quadratic loss. EP itself, a significant contribution to local learning, is a one-term modification of an energy combined with the implicit function theorem. Simple constructions can carry substantial consequences.

Q4. Why pixel space. We use "neuromorphic" in the sense of efficient physical computation rather than brain mimicry. Iterative image generation is a large and growing workload, and its hundreds of sequential velocity evaluations per sample are exactly what a physical relaxation could replace. We work in pixel space because Energy Matching does, which makes the correspondence and the quality comparison direct. Energy Matching has since been applied in a learned latent space (Barco et al., 2026), and latent generation is a natural next step (with a decoder also trained by EP) that would also reduce the number of nodes relaxing at each step (now noted in the "Discussion and limitations" section).

We hope these revisions address the reviewer's concerns. If so, we would be very grateful if the reviewer would consider revising their score.

AuthorsOpenReview ↗

Follow-up on our response

We would like to thank the reviewer again for their questions. As the discussion period closes soon, we wanted to check whether our response and the revised paper address their concerns about novelty, related work and readability. We would be happy to clarify anything further.

Thank you for the detailed response and revision. The expanded related work fully addresses my first question, and the added Figure 1 and conclusion improve readability.

However, my concern about novelty remains. As Appendix M acknowledges, PC equilibria carrying backpropagated errors is a known result. The injection term is closely related to an EP/PC nudge whose cost is the network output itself, read out at the input layer. The linearised nudge is adopted from Dual Propagation, and the new bias analysis is specific to GradEP with a quadratic loss. I recognize the empirical value of reaching CIFAR-10 at backpropagation quality, but the contribution lies mainly in combining and scaling existing components rather than in a new conceptual insight.

I will therefore maintain my score.

AuthorsOpenReview ↗

Thank you for your response

Thank you for engaging with our response and for re-reading the revision. We are glad the related work and readability now address your concerns. We agree that PC equilibria carrying an output error down through the network is known, and the paper credits this. Our view is that the injection term itself, a linear term on the network’s scalar output with no target, is new, as is its consequence: the PCN’s own energy becomes the generative potential, so sampling itself becomes a relaxation, which earlier analyses did not consider. We also appreciate the reviewer’s recognition of the empirical value of reaching CIFAR-10 at backpropagation quality, a level that backprop-free generation had not previously reached. Thank you again for a careful and constructive review, which has made the paper considerably stronger.

Reviewer_L3ioOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PotentialEP explores equilibrium-based learning for energy-based generative models. It converts a feedforward potential network into a predictive coding network whose equilibrium forces approximate the potential’s gradient, enabling generation through relaxation. For training, a spring constraint encodes the velocity as an equilibrium displacement, allowing Equilibrium Propagation to estimate parameter gradients from local measurements. Linearized nudging improves numerical stability and gradient accuracy.

Critical strengths
  • Clear theoretical foundation: Establishes an O(γ)O(\gamma) correspondence between equilibrium forces and the original potential gradient.
  • Pretrained-model compatibility: Existing Energy Matching models can reuse their weights without retraining, with negligible sampling-quality loss.
  • Accurate learning: EP updates closely match backpropagation directions, achieving cosine similarity of 0.9999.
  • Competitive generation quality: Reaches CIFAR-10 FID 3.64, close to the BP baseline of 3.54.
Critical weaknesses
  1. The “backpropagation-free” claim has a restricted scope.
    Although parameter updates are formulated through equilibrium measurements, the actual implementation still relies on reverse-mode differentiation to find equilibria in error coordinates. In the reported single-sweep setting, this computation is algebraically equivalent to a backward pass. Furthermore, a feedforward network generates second-stage negative samples using autograd. Consequently, the experiments validate the equilibrium-based learning rule, while a complete pipeline operating without backpropagation remains undemonstrated.

  2. Equilibrium correspondence does not establish practical physical computation.
    The theoretical results assume sufficiently accurate equilibria, but reaching these equilibria efficiently is a central unresolved issue. The authors report severe ill-conditioning for direct state-coordinate relaxation and therefore use a more tractable error-coordinate solver. Having identical stationary points does not imply that these dynamics have comparable convergence rates.

  3. The method has substantial computational overhead in its demonstrated setting.
    On GPUs, an EP training step costs approximately 16 times as much as a backpropagation step; implicit differentiation costs approximately five times as much. This limits the method’s immediate practicality and constrains the scale of experiments the authors can perform.

  4. The experimental design provides limited statistical confidence.
    Each training method is evaluated using only one run, with different initializations and random training realizations.

Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer L3io

We thank the reviewer for the clear and precise summary of our weaknesses. Each point has led to new measurements or clarifications in the revision.

1. The scope of "backpropagation-free". We agree with the reviewer's reading, and the paper now states the scope explicitly (the "Scope of the claims" appendix): our experiments show that the weight updates need no backward pass, not that the whole training loop runs without one. We have since closed both gaps the reviewer identified as far as our compute budget allows.

  • The solver. Our error-coordinate solver is a simulation device, and the paper's claims concern the equilibrium, not the route to it. The new appendix "Training with local state dynamics" runs the local state-coordinate dynamics a physical substrate would use, to convergence, with no solver shortcut and, for the CNN, at one node per weight matrix. They reproduce the solver's velocities and EP updates to a cosine similarity of 1.000000 (relative error at most 4×10−34\times10^{-3}), on both a backprop-trained UNet+ViT and an EP-trained CNN. Most importantly, a CNN trained from scratch by EP with these dynamics follows the solver's loss curve (see Figure 3), with per-step updates agreeing to a median cosine of 0.99996.
  • The feedforward twin. It only generates phase-2 negatives, and no parameter gradient passes through it. We now test it rather than assume it. Replacing it with relaxation moves the negatives by a median relative distance of 2.4×10−32.4\times10^{-3}, within a factor of two of rerunning the twin's own sampler on identical inputs (1.3×10−31.3\times10^{-3}, from TF32 arithmetic alone) and about seventy times less than changing the sampler's noise seed, and leaves the contrastive update unchanged to a cosine of 0.999998.

2. Equilibrium correspondence versus convergence rates. We fully agree that shared stationary points do not imply comparable convergence rates, and we now measure the rates directly. Heavy-ball local dynamics need about 25×25\times the solver's sweeps on the CNN and over 103×10^3\times on the CIFAR UNet+ViT. The cost follows the conditioning κ\kappa of the relaxation, which depends strongly on the architecture. On physical hardware the step size disappears but the conditioning does not: settling takes at least κtmin⁡ln⁡(1/ϵ)\kappa t_{\min} \ln(1/\epsilon), or κtmin⁡ln⁡(1/ϵ)\sqrt{\kappa}t_{\min} \ln(1/\epsilon) with second-order dynamics. With nanosecond node time constants this gives under a microsecond per relaxation for the CNN and tens of microseconds for the CIFAR UNet+ViT (κ\kappa of order 10710^7), so about 10 ms per CIFAR-10 sample, against 45 s on an A100 at batch size one. The "Towards analog implementation" appendix also identifies routes to better conditioning, including per-node time constants (a measured threefold reduction in κ\kappa).

3. Computational overhead on GPUs. We agree, and we say so in the paper: on a GPU, backpropagation remains the cheaper way to train this model. The 16×16\times is the cost of simulating a relaxation on hardware built for single-pass computation. The case for PotentialEP rests on substrates where the relaxation is physical, whose projected latency is given above. The inference result also needs no EP training at all, since backprop-trained models load into the PCN unchanged and match their original FID to within 0.04 at CIFAR-10 and ImageNet-32 scale.

4. Statistical confidence. We agree that more training runs would strengthen the paper. A full two-phase EP run at CIFAR-10 scale exceeds our remaining budget (the original Energy Matching paper also reports single runs). To test robustness within that budget, we measured what the training rule depends on, the agreement of EP's parameter update with the backpropagation gradient. It holds at a cosine similarity of 0.9997±0.00010.9997 \pm 0.0001 (minimum 0.9994) across nine phase-1 checkpoints, at 50k, 100k and 145k steps of all three training runs, with five independent batch draws each. The estimator is therefore consistent throughout training and whichever rule trained the weights. Two EP contrastive phases from the same checkpoint, at the same nudge prefactor reached by different settings, also reach FIDs of 3.33 and 3.64, both within 0.3 of backpropagation's 3.54. The paper also states the FID measurement's own resolution (about 0.2) and treats smaller differences as unresolved.

We hope these additions resolve the reviewer's concerns, and we would be grateful if the reviewer would consider raising their score.

Thank you for your response. You have addressed all my questions, and I will maintain my initial positive score.

AuthorsOpenReview ↗

Thank you for your response

Thank you for reading our response and for your positive assessment. We are glad the revision addresses your questions.

Reviewer_6NG2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors outline a novel application of EP to image generation via energy matching. They demonstrate that this performs at imagenet-32 scale while maintaining near-backprop performance. In my opinion this is a very well motivated use of EP since indeed, as the authors mention, inference can be achieved with significant speedup and significant reduction in energy consumption on devices whose inference is a physical relaxation, and regardless of challenges related to training EP at scale , backprop-trained models can be likewise loaded and efficiently sampled from analog hardware directly.

Critical strengths

Overall I really enjoyed this paper. I think it's a clever and very sensible application of EP where the approach clearly suits an analog substrate. Generally very well argued and tight presentation.

Critical weaknesses

While this may be the first generative model using specifically CIFAR-10, an EP based generative model from the same team you cite in your asynchronous relaxations section (kerjan et al) was applied on CELEB-A with good results (Hoier, 2026: Training a convergent energy transformer with equilibrium propagation). I would acknowledge this explicitly. I would also place greater emphasis on the similarities and differences of this approach (EP on a PC energy) with the Kerjan paper, since theirs is the most similar approach to yours that I am aware of in the EP lit architecture wise.

I took issue with some of the writing which gives sections of the paper away as being AI-generated. "Table 6 gives the settings shared by all arms, and the relaxation settings of the two PCN arms. The three arms of Section 5..." In my experience this language of "arms" is very typical of claude code and I have almost never seen it in use outside of an LLM, certainly not common in academic/scientific writing. I understand that these are boring implementation details and not the central bits of the paper, but would recommend rephrasing (by hand) in a more scientific and less engineering/LLM-session tone. Perhaps this style is becoming scientific convention , but in my opinion it would be better if researchers still made an effort to adhere to more human standards of scientific communication , particularly in a conference like this. In general though much of the writing is fine and I enjoyed the paper.

A more significant shortcoming of this work is the lack of attention given to realistic implementation on analog. I would like to see an additional section that speaks directly to realistic implementation details, implementation barriers (there is a lot of literature dedicated to this), and how they might be circumvented. I think alongside these additions, a detailed account of what advantages (projected energy/ inference time savings) could be expected from analog implementation would also significantly strengthen this paper.

Questions for authors

None, really.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 6NG2

We thank the reviewer for the generous and constructive review, and we are glad they enjoyed the paper. Each of the three suggestions has led to a concrete revision.

Høier et al. (CET on CelebA). Thank you for pointing us to this work. It is now cited in the "Background" and discussed in "Related work". CET trains a convergent Energy Transformer by EP for masked completion of 120×120120\times120 CelebA faces, matching truncated backpropagation through the relaxation in reconstruction error. It maps a partial image to a single deterministic completion under an MSE loss, whereas PotentialEP learns a velocity field from which new images are sampled. Our claim is therefore to the first EP-trained model for unconditional generation at CIFAR-10 scale, not the first application of EP to images. We see the two as complementary demonstrations that EP scales beyond classification, and we now present them side by side.

Kerjan et al. (EP on PC energies). We agree this is the closest work architecturally, and it now opens the "Related work" section. We share their energy family and their even/odd relaxation, and their ImageNet result (13.23% top-5 error against 12.2% for backpropagation) is strong evidence that EP on PC energies scales. The differences are in what is trained and where the relaxation is used:

  • Their nudge displaces the output layer towards a label, and their free equilibrium is a forward pass. Ours trains the gradient of the network's scalar output with respect to its input. The injection term places that gradient in the equilibrium errors, and GradEP's tether makes it trainable, so both sampling and training run on the relaxation.
  • Training a gradient loss with EP needs a nudge that is both strong enough to resolve in finite precision and free of bias. The linearised nudge and the joint γ\gamma–β\beta window it opens (usable β\beta from 3 to 10310^3 instead of a single value) are what make this work at CIFAR-10 scale.

As in their work, a PC energy inherits any feedforward architecture. Here that lets us train exactly the UNet+ViT of the backpropagation-trained Energy Matching baseline, so the quality comparison is like for like and backprop-trained checkpoints load into the PCN unchanged.

Writing. Thank you for flagging this, and for the specific example. "Arms" has been removed throughout, and we have reworked the prose with a more conventional scientific tone. We also added Figure 1, an annotated schematic of the PotentialPCN for additional clarity, and a conclusion.

Realistic analog implementation. We agree this was the most significant gap, and a new appendix section titled "Towards analog implementation" addresses it:

  • Circuit picture. With normalisation and attention removed (FID 3.51), the network needs only weighted sums, pointwise nonlinearities and integration. States are capacitor voltages, weights are crossbar conductances, and Kirchhoff summation supplies the forward and backward drives.
  • Barriers and how to circumvent them. Simultaneous transposes (an error-node design in which one resistor carries WW one way and W⊤W^\top the other, time-multiplexing, or matched arrays), signed weights (differential pairs), activation derivatives (a switch for ReLU, or a matched element pair for smooth activations), convolution area, the known obstacles to writing weight updates, and timescale separation between hidden nodes and xx.
  • Settling time. It is bounded below by κtmin⁡ln⁡(1/ϵ)\kappa t_{\min} \ln(1/\epsilon), or κtmin⁡ln⁡(1/ϵ)\sqrt{\kappa} t_{\min} \ln(1/\epsilon) with second-order dynamics, and we identify routes that reduce κ\kappa, such as per-node time constants (a measured threefold reduction).
  • Projected savings. Latency: at nanosecond node time constants a CIFAR-10 sample takes about 10 ms, against 45 s for our unoptimised sampler on an A100 at batch size one. Energy: measured analog in-memory chips already reach about 30×30\times an A100's peak efficiency, and design studies project savings of a factor of 10310^3 or more. The advantage is clearest for single-stream and edge generation.

A companion appendix titled "Training with local state dynamics" shows that the local dynamics such a circuit would run reach the same equilibria as our solver and train a model at one node per weight matrix.

We hope these revisions address the reviewer's concerns. If they do, we would be grateful if the reviewer would consider raising their score.

I thank the authors for addressing my comments. I am satisfied that my concerns have been largely addressed. In the spirit of full transparency I still feel the recommendation of acceptance (3) is appropriate since the work , though thorough and clear, is not exceptional in its novelty and the proposed analog application remains largely hypothetical and prospective. I will however take my time to re-read with an open mind the paper in its entirety inc. proposed additions. For now I have no additional concerns.

AuthorsOpenReview ↗

Thank you for your response

Thank you for taking the time to read our response and for re-reading the revised paper. We agree that the analog application is still prospective, and we have tried to be clear about that in the revision. Working with hardware groups towards a physical realisation is the focus of our follow-up work. Thank you again for your support of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all three reviewers for their careful and constructive reviews. We were encouraged that 6NG2 found the approach "a clever and very sensible application of EP where the approach clearly suits an analog substrate", that L3io highlighted its "clear theoretical foundation" and "pretrained-model compatibility", and that YTKD saw it as addressing "a timely and compelling problem" with "a practical application scenario where such a model provides tangible benefits".

The reviews converged on four requests, and the revised PDF addresses each. Substantive additions are marked in blue, and the prose has been revised throughout.

1. Analog implementation (6NG2, L3io). A new appendix section titled "Towards analog implementation" gives a concrete possible circuit picture (capacitive nodes coupled through resistive crossbars), the device requirements with known options for each (transposes, signed weights, activation derivatives, convolution area, weight writes, timescale separation), settling time and conditioning, precision, and projected latency and energy. At nanosecond node time constants, one CIFAR-10 sample would take about 10 ms, against 45 s for our implementation of the same sampler on an A100 at batch size one. Single-stream generation of this kind, as in real-time use at the edge, is where we expect analog hardware to gain most over GPUs.

2. Local dynamics reach the same equilibria (L3io). A new appendix section titled "Training with local state dynamics" runs the local state-coordinate dynamics a substrate would use to convergence, including at one node per weight matrix. They reproduce both our solver's velocities and EP parameter updates to a cosine similarity of 1.000000, and a model trained from scratch with them follows the same loss curve. The appendix also measures the convergence rates directly.

3. Positioning and novelty (6NG2, YTKD). The "Related work" section is substantially expanded, with explicit comparisons to Kerjan et al. (EP on PCN energies), Høier et al. (EP on a convergent Energy Transformer), the PC–backpropagation equivalences including the infinitesimal-inference-limit analysis of Millidge et al. (2022) that YTKD raised, FlowEqProp, other local and backprop-free generative models, and recent PC advances. The "Scope of the claims" appendix now states precisely what is new relative to prior work, and a new conclusion in the main text summarises the contributions.

4. Readability (6NG2, YTKD). We have reworked the prose throughout, removed engineering jargon such as "arms", and added Figure 1, an annotated schematic of the PotentialPCN. The tenth page goes to the expanded related work, a conclusion and less compressed prose.

We also added two robustness checks for the single-run concern. First, EP's updates match backpropagation's to a cosine similarity of 0.9997±0.0001 across nine checkpoints (50k, 100k and 145k steps from each of the backpropagation, implicit-solve and EP runs), with five independent batch draws of 512 interpolants each (the "Training fidelity" appendix). This does not replace reseeded training runs, but it shows that the property EP training depends on holds throughout training, whichever rule produced the weights. Second, a further EP phase-2 run, keeping the phase-1 relaxation settings and changing only the nudge strength (β=7.5\beta = 7.5), reaches an FID of 3.33, on par with backpropagation's 3.54 under the same protocol. The revision now reports it as the main EP result. Successful EP training is therefore not tied to one finely tuned setting.

We are very grateful for the reviewers' time. Their comments have made the paper clearer and its case for analog hardware considerably more concrete.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.