Coherent Predictive Coding: Stabilizing Latent-State Inference and Local Parameter Learning

Predictive coding (PC) is a biologically inspired alternative to backpropagation that enables local and parallelizable learning by iteratively inferring latent states and learning parameters from layerwise prediction errors. However, forming useful prediction errors within a limited inference budget remains challenging. We address this problem as two connected stages: latent-state inference, which forms prediction errors, and local parameter learning, which translates them into parameter updates. In Standard PC, latent-state updates depend on learned Jacobians evaluated at the current latent states. After inference, anisotropic activation statistics can distort how local parameter gradients are expressed as block-output changes. We propose Coherent Predictive Coding (CoPC), which fixes feed-forward reference activations $a_l$ and represents latent states as deviations $d_l=h_l-a_l$, yielding $e_l=d_l-d_{l-1}$ so that intermediate inference uses fixed identity coupling rather than learned block Jacobians evaluated at the evolving latent states. After inference, CoPC computes local parameter gradients from the final prediction errors and preconditions each selected linear projection using its uncentered activation second moment to reduce function-space distortion. Across MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, CoPC achieves competitive predictive-coding performance and reaches $27.12\pm 1.01\%$ test accuracy on CIFAR-100, exceeding the strongest predictive-coding baseline in our comparison by 3.25 percentage points. Controlled analyses further show that CoPC produces more consistent latent-state update directions and that second-moment preconditioning improves the alignment between final prediction errors and the induced block-output changes.