Transformer Neural Processes with growing receptive field for uncertainty prediction over clustered observations

Neural Processes (NPs) are neural-network-based and Gaussian Process (GP)-inspired models for uncertainty prediction, yet most NP backbones are designed and evaluated under approximately uniform observations. Real-world data are rarely so ideal: measurements collected across the Earth are often clustered, with dense sampling near accessible regions and gaps elsewhere. Recent transformer NPs restrict attention to small local windows for efficiency and report little loss in accuracy; we show, however, the size and structure of the receptive field can affect predictive performance. We propose a growing-window processor that enlarges the attention window layer by layer and appends a global layer to capture both short-range and long-range dependencies. On synthetic Gaussian random fields and real-world climate data (ECA&D), our model achieves better predictive log-likelihood on clustered tasks than the baselines with local windows, indicating that clumped observations benefit from combining local and global receptive fields.