Direction, Not Bandwidth: What Limits Spectral Neural Operators
Abstract
Neural operators are adopted for accuracy that survives a change of discretization, and Fourier neural operators demonstrate it on turbulent flow, reporting that the network recovers the frequencies it discards. We show that this recovery breaks down on heterogeneous solids, where the operator damps a high frequency no more than a low one. On the elastic Green's operator of homogenization, only the model that keeps every mode is accurate: keeping sixteen modes per axis gives a relative error of 0.44, against 0.026 for keeping all of them. We show this limitation belongs to the architecture rather than to training, and remove it by changing what the spectral weights represent rather than how they are learned. First, we prove that the error of any linear mode-truncating architecture is bounded below by how much the operator varies with direction. The bound is independent of the number of modes, the width and the depth, and can be computed before training. Second, we measure it on the elastic Green's operator, its anisotropic form, and the Stokes projection. Trained nonlinear models sit at the bound, truncated ones at or above it and isotropic ones within 4%, and neither added width, nor longer training, nor a loss reweighted toward high frequencies moves them off it. On a smoothing operator, where the bound is near zero, the same models are accurate and an isotropic parameterization wins instead. The direction dependence is therefore the cause. Third, we replace the table of per-mode weights with an expansion in spherical harmonics of the wavevector direction, applied at every frequency. Six coefficients then reproduce the Stokes projection exactly, at any resolution, where no finite mode table does. Placed inside a standard homogenization solver, and initialized to reproduce that solver exactly, 81 learned coefficients halve its iteration count at a phase contrast of 100 and cut it by more than 2.5× at 1000. The analytic operator at the centre of such solvers can therefore be replaced by a learned one that reaches the same accuracy in fewer iterations.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.