Product Spectrum Neurons: Making Interaction Order Addressable

Neural networks multiply features everywhere, yet interaction order usually remains entangled with width, depth, and gating rather than freely addressable. The Product Spectrum Neuron (PSNeuron) computes, in each of its \(R\) learned bases, elementary symmetric polynomials of degree \(2\) through \(K\) over \(G\) disjoint slots. Every order becomes its own channel, with three exact properties: on Rademacher inputs channel \(j\) carries Walsh degree \(j\); increasing \(K\) at fixed geometry appends orders without disturbing lower ones; and any order can be masked in a trained model without retraining or modifying learned parameters. Across three tabular tasks, one common PSNeuron architecture with \(K=4\) fixed in advance, a validation-selected layout, and task-specific learning rates removes 34.7% to 51.7% of the trainable parameters of the selected dense references while remaining within 0.44 points of their accuracy. At \(K=2\), narrow DINOv3 heads outperform an exact-parameter additive control in 69 of 75 paired seeds across 15 dataset and width settings, and on 20 fresh seeds a \(K=4\) DeltaNet mixer beats an exact same-slot additive twin with 48 fewer order coefficients and a conditional GELU control on associative recall after Holm correction. On controlled teachers, exact PSNeuron witnesses establish representability independently of training, order 6 content is recovered reliably near its target degree, and same-model spectra and masks verify the planted decomposition in recovered models and localize missing upper-order content in failed runs. PSNeuron turns interaction order into an addressable experimental coordinate of a neural site whose representability, recovery, and functional use can be examined within the same trained model.