Symbol-conditioned Adaptive Fourier Operator Transformer for Large-Scale PDE Pre-Training

Neural operators have emerged as a promising paradigm for solving PDEs and building PDE foundation models. However, existing architectures often suffer from spectral bias and struggle to capture high-frequency dynamics, limiting their capacity to model complex physical systems. To address these challenges, we introduce the Symbol-conditioned Adaptive Fourier Pre-training Operator Transformer (SAFPOT). First, we define multi-level frequency bands to represent heterogeneous spectral regions, enabling full-frequency learning. First, the multi-level frequency bands are defined to represent heterogeneous spectral regions, enabling full-frequency learning. Second, we design a novel Symbol-conditioned Adaptive Fourier (SAF) module, which dynamically modulates low-, mid-, and high-frequency bands via a frequency gate network. Together, these designs allow the model to adapt its spectral modeling strategy to the characteristics of each governing equation and effectively capture multi-scale physical dynamics. Finally, we pre-train models with parameters from 40M to 0.5B on 6 PDE datasets. In data-oriented evaluation, pre-trained SAFPOT with only 28M activated parameters surpasses fully fine-tuned baselines and further reduces L2RE by up to 71.75% after PDE-specific fine-tuning. In physics-oriented evaluation, SAFPOT reduces the average fRMSE by up to 59.4% compared with the strongest baseline and substantially reduces errors across each frequency band.