Generative Energy-Based Predictive Coding Networks

Predictive-coding networks (PCNs) perform inference by relaxing hidden activities to minimise layer-wise prediction errors. Although this process is commonly described in terms of energy minimisation, the resulting state objective does not by itself define a probability distribution over sensory inputs and therefore cannot directly support input-space generation. In this work, we propose the Generative Energy-Based Predictive Coding Network (GEB-PCN), which turns the state energy of a forward PCN into a trainable input-space energy model. We define a joint energy over inputs and hidden activities by combining layer-wise prediction errors with an output energy, and obtain the energy of an input by evaluating this objective at its inferred hidden state. Contrastive learning uses finite-step inferred states to distinguish data from model-generated samples, without requiring an explicit top-down generative hierarchy. To generate samples, we develop a coupled PC–Langevin procedure that jointly refines hidden activities and sensory inputs. Under the original benchmark configurations, GEB-PCN attains the lowest domain FID among all compared generators on MNIST and Fashion-MNIST, well below the best-performing baselines. FiLM conditioning of the joint energy further demonstrates class-conditional generation for generative PCNs.