acceptodds
Under review as a conference paper at ICLR 2027

Generative Energy-Based Predictive Coding Networks

Abstract

Predictive-coding networks (PCNs) perform inference by relaxing hidden activities to minimise layer-wise prediction errors. Although this process is commonly described in terms of energy minimisation, the resulting state objective does not by itself define a probability distribution over sensory inputs and therefore cannot directly support input-space generation. In this work, we propose the Generative Energy-Based Predictive Coding Network (GEB-PCN), which turns the state energy of a forward PCN into a trainable input-space energy model. We define a joint energy over inputs and hidden activities by combining layer-wise prediction errors with an output energy, and obtain the energy of an input by evaluating this objective at its inferred hidden state. Contrastive learning uses finite-step inferred states to distinguish data from model-generated samples, without requiring an explicit top-down generative hierarchy. To generate samples, we develop a coupled PC–Langevin procedure that jointly refines hidden activities and sensory inputs. Under the original benchmark configurations, GEB-PCN attains the lowest domain FID among all compared generators on MNIST and Fashion-MNIST, well below the best-performing baselines. FiLM conditioning of the joint energy further demonstrates class-conditional generation for generative PCNs.

Primary area:
applications to neuroscience & cognitive science
Keywords:
predictive coding, energy-based models, generative modeling, langevin dynamics, contrastive learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_P4obOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces GEB-PCN, which combines forward predictive-coding mismatch penalties and an output potential into a joint energy over images and hidden activities. Training contrasts positive and negative finite-state energies while stopping gradients through hidden-state inference. A coupled sampler updates hidden activities deterministically and images with noisy energy-gradient steps. The paper explicitly distinguishes this implemented finite-step system from the equilibrium profile energy and exact Gibbs sampling. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 report image-quality metrics against predictive-coding, EBM, and latent-variable baselines. Further analyses examine class conditioning, PC-force interventions, replay versus noise starts, exact hidden-state profiling, and joint-versus-separate sampling cost.

Critical strengths
  1. The method gives a concrete, implementable route from a forward-PC state objective to image synthesis. Equations (7)-(10) and Algorithms 1-3 specify the joint energy, contrastive update, and carried-state sampler. The authors correctly distinguish a state objective from a normalized input model and do not assume that finite hidden inference reaches equilibrium. The conditional MNIST experiment also demonstrates meaningful generation from noise, rather than relying entirely on replay (Appendix E, Table 13).

  2. The sampling analysis is unusually candid and informative. The grayscale results aggregate five training seeds and include KID and PRDC rather than FID alone. Sections 5.1-5.3 and Appendix F separate sampling-time force interventions from claims about the benefit of training with PC. The long-chain, exact-profile, and paired timing controls show both benefits and limitations: joint updates are 3.16-3.92 times faster than the tested separate-eight-state-update sampler, but they do not establish a general equal-time advantage. These distinctions improve the credibility and scientific usefulness of the study.

Critical weaknesses
  1. The CIFAR-10 headline does not establish synthesis independent of the training-image population. The selected continuation refreshes particles using perturbed training images (Appendix C.4), and Appendix D.6 states that at least 9,207 of 10,000 selected-buffer entries have received such a refresh. The resulting main-table FID of 62.93 therefore measures refinement of a data-refreshed starting population. In the separate fixed-reference control, stored-particle test FID is already 82.53 before sampling and improves to 71.88, whereas a fresh 5,000-image noise-start evaluation at the selected 500-step horizon has test FID 176.50. These are different protocols, so their numbers must not be subtracted as if they were a paired comparison. Nonetheless, the initialization dependence is large and explicitly documented. The EBM comparison also changes training duration and replay policy, and has only one training seed (Appendices C.6 and D.6). Thus the RGB experiment does not yet isolate the model's generative benefit from the replay/data-refresh recipe. A shared-start, shared-replay comparison with independent seeds and train/test references, or a clear main-table separation of data-conditioned refinement from noise-start synthesis, is needed before this result can carry substantial weight for acceptance. This is a limitation of the demonstrated claim, not an accusation of concealed data use.

  2. The distinctive contribution of internal PC is not established relative to an otherwise matched energy model. The large Fashion-MNIST gap to the EBM in Table 1 compares a hybrid convolutional-MLP MSE-head GEB-PCN with a scalar-head MLP EBM (Section 4.1; Appendices C.1 and C.5), so architecture and head design can contribute to the gain. Existing ablations do not resolve this: removing the head after training is not a retrained necessity test, the Appendix F four-cell comparison uses different selected recipes, and the genuinely paired conditional PC-on/off cohort in Appendix E.1 favors PC-off in label accuracy and macro FID (0.9348 versus 0.8986; 3.782 versus 6.371). That last cohort has five seed pairs and a test-informed recipe; it does not prove that PC is universally harmful or settle the unconditional comparison, but it makes a controlled positive demonstration particularly important. Please compare internal PC-on/off with the same backbone, output potential, replay protocol, selection budget, and training seeds in the primary unconditional setting, and include the sampling-cost trade-off. Otherwise, the evidence supports an interesting structured EBM construction and sampling study more clearly than a practical advantage attributable to internal predictive coding.

Questions for authors
  1. Clarification affecting the RGB claim: Can you report GEB-PCN and the no-PC EBM under the same frozen starting population, replay/refresh policy, training-selection protocol, and sampling budget, including K = 0 and held-out-reference results? Which conclusion about independent RGB synthesis remains if data-refreshed starts are excluded?
  2. Clarification affecting component attribution: Are matched unconditional PC-on/off runs available with the same hybrid/MSE backbone and prospectively fixed selection protocol? In light of the paired conditional ablation, what measurable benefit do the internal states provide beyond the energy head and replay mechanism?
  3. Clarification affecting the advertised conditional comparison: Section 4.3 calls conditional performance highly comparable to the VAE, but Table 13 reports macro cFID 3.121 versus 1.277 and global FID 0.743 versus 0.386, with lower label accuracy for GEB-PCN. Will you revise the main-text comparison to foreground this fidelity gap and specify the narrower benefit of the PC construction?
Reviewer_rfobOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a new energy-based framework, linking the prediction errors of a forward PCN with a trained output energy. Using contrastive learning at inferred hidden states with a Langevin-based sampler, the authors define a shared energy function that couples hidden-state inference and input-space generation. It enables generation withoutan explicit top-down decoder. The authors also invest igated replay initialization, PC force, and joint-update efficiency as partof sampling-time analysis. On three controlled datase ts (MNIST, Fashion-MNIST and CIFAR-10), the authors show that their method achieves the lowest FID compared to the evaluated baselines, within the evaluated configurations. The framework also allows class-conditional generation thanks to FiLM conditioning.

Critical strengths

1- Smart and consistent design for working with neural networks through energy-based methods. The approach is interesting because it exploits the network's internal structure and could help us better understand how these networks work, even at a relatively small scale. The paper also offers a way to perform input-space generation without constructing an explicit top-down decoder hierarchy, which is in teresting. 2- The experimental design is careful and well documented, making the reported effects clearer and more reliable (five seeds, successive experimental steps, and interesting comparisons to observe the improvements). The framework is explored through several experiments, including FiLM conditioning and architectural variations.

Critical weaknesses

1- Although the experiments are quite comprehensive, I find the baselines a bit limited. In particular, EBM and GEB-PCN differ in their training duration/replay policy/checkpoint selection on CIFAR, making it difficult to isolate the benefits of the PC mechanism. The selected baselines provide a relatively narrow perspective on the generative performance of the framework. More importantly, the conditional ablation proposed in Appendix E report that disabling the PC component actually improves generation quality and label accuracy in the evaluated configuration. Although we can not generalize it to all GEB-PCN variants, I believe it raises an important question about the contribution of the central mechanism and deserves further investigation. Given its relevance to the proposed method, this limitation should also be discussed more explicitly in the core of the paper. This limits the strength of the empirical conclusions and the impact of the comparisons, despite the careful design of the proposed method.

2- The major comparisons are done on very toy examples (MNIST and Fashion-MNIST) which reduces the reader ability to see the real impact. On CIFAR-1°, the generative quality remains limited in absolute term despite outperforming the selected baselines. This raises the question of scalability to more complex settings.

3- Furthermore, the reported CIFAR FID of 62.93 uses a stored-particle population in which at least 9,207/10,000 buffer entries had received a noisy-training-image refresh. In a separate fixed-checkpoint, test-reference evaluation, the authors report an FID of approximately 71.88 from stored particles compared with 256.80 from noise after 200 updates. Although these figures are not directly comparable with the main-table result, they highlight a true dependence on initialization. Consequently, it remains unclear to what extent the reported performance demonstrates independent generation rather than refinement of data-informed samples. This also limits the conclusions that can be drawn about generalization to more challenging generative tasks.

Questions for authors

see weaknesses

Reviewer_55vmOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

GEB-PCN combines the prediction errors of a forward predictive-coding network with an output energy to construct an input-space energy model without an explicit top-down decoder. It uses contrastive training at inferred hidden states and generates images through coupled hidden-state and Langevin input updates. Experiments cover MNIST, Fashion-MNIST, and CIFAR-10, alongside class-conditional generation using FiLM.

Critical strengths
  1. A clear connection between forward predictive coding and energy-based generation. The framework connects hidden-state inference and image generation through a shared energy function, providing a concrete way to generate images without an explicit top-down decoder.
  2. Informative analysis of sampling behavior. The paper examines how the PC force, replay initialization, and sampling length affect generation. It also demonstrates that joint updates achieve similar FID with lower sampling latency than the tested separate-update procedure.
Critical weaknesses
  1. Improved sample quality does not necessarily imply greater diversity. In the PRDC results, GEB-PCN has higher Precision than MCPC and the VAE but lower Recall, with a particularly large gap on Fashion-MNIST. Although the authors acknowledge this trade-off, low FID and high IS alone do not establish superior overall generative performance. Further analysis is needed to identify which types of data are underrepresented and whether diversity within each class is preserved.
  2. Short replay-based sampling may limit exploration. Starting from stored samples and applying only a limited number of updates may refine existing patterns without adequately exploring others. Adding these samples back to the buffer could perpetuate that limited exploration. The paper analyzes how FID changes with initialization and the number of sampling steps; reporting Recall and Coverage under the same conditions would help determine whether the observed diversity limitation stems from replay and short sampling.
Questions for authors
  1. Which classes or types of samples account for GEB-PCN’s lower Recall than MCPC and the VAE, particularly on Fashion-MNIST? Is diversity within each class preserved?
  2. How do Recall and Coverage change with replay versus noise initialization and with longer sampling chains? Can these comparisons clarify whether short replay-based sampling contributes to the observed diversity limitation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.