Spatially Adaptive Convolutions via Attention-Routed Expert Mixtures

Convolutional autoencoders have been widely used for image reconstruction due to the local receptive fields and inductive biases of convolutional operators. However, conventional convolution applies the same learned transformation across spatial locations, limiting the ability of a single operator to adapt to heterogeneous local structures. We investigate whether this limitation can be addressed by learning a spatially varying composition of distinct convolutional transformations. We propose a mixture-of-experts (MoE) based adaptive convolutional autoencoder in which a fixed bank of structurally heterogeneous three-dimensional convolutional experts provides distinct transformations, while a spatial routing mechanism determines their contributions at different locations. In the encoder, localized window representations are processed by an attention module to produce spatially varying expert weights, which are interpolated to form a dense routing field. The resulting formulation retains the parameters of the individual experts while allowing their effective composition to vary spatially. We evaluate the framework on three-dimensional magnetic resonance imaging (MRI) reconstruction using the IXI-T1 dataset, focusing on the behavior of the learned experts and their spatial routing. The individual experts exhibit systematically different responses to image structure and spatial frequency, while routing redistributes their contributions across the feature map and retains a soft, non-uniform mixture rather than hard expert assignment. These observations are consistent with functional specialization emerging among heterogeneous convolutional operators through spatial routing.