Native Multi-Dimensional Subquadratic Operators via Input Dependent Long Convolutions

Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions abandon the global, input-dependent receptive field, while recurrent models require rasterizing images, volumes, and PDE grids into an ad-hoc $1\mathrm{D}$ scan order that violates their spatial structure. We introduce HyenaND, an operator that acts directly on native $\mathrm{ND}$ data in its intrinsic geometry and recovers a global, input-dependent receptive field at subquadratic cost. It uses an implicit, input-dependent parameterization of the multi-dimensional convolutional kernel. We provide a CUDA implementation, nSubQ, which fuses the FFT-convolution path to turn HyenaND's $O(N\log N)$ scaling into wall-clock speedups. Across long-context genomics, computer vision, medical imaging, and PDE modeling, pure HyenaND stacks match the accuracy of strong attention-based baselines, while hybrid configurations that interleave HyenaND and attention layers outperform both pure attention and strong recurrence-based hybrids.