Parameter-Efficient Learning with Hybrid Volterra Neural Networks

Despite the flexibility of multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs), they learn nonlinear frequency interactions indirectly through successive layers, which can increase parameter requirements. For non-polynomial pointwise activations such as ReLU in these networks, a globally exact finite-order polynomial representation is generally unavailable. To address this limitation, we propose the Hybrid Volterra Neural Network (HVNN), which assigns selected spectral components to an activation-free Volterra branch and sends the remaining spectrum to a neural branch. The Volterra branch directly parameterizes nonlinear responses through finite-order polynomial expansions in the frequency domain, making nonlinear frequency coupling explicit. We further propose spectral interaction sparsification to restrict computation to a budgeted set of valid sum- and difference-frequency pairs, thereby limiting evaluated nonlinear interactions. Thus, HVNN constrains model complexity, targeting improved predictive performance with a modest parameter count. HVNN-MLP and HVNN-CNN variants are evaluated on four nonlinear system-identification, three image-denoising and two image-classification datasets to demonstrate their ability across diverse predictive tasks and to achieve a favorable accuracy–parameter trade-off. Our HVNN-CNN reaches PSNR values of 30.7351 and 26.8766 on MNIST and Fashion-MNIST denoising with 65.6K parameters, compared with 24.1188 and 22.3718 with 79.0K parameters for CNN. The code is available at this link.