Mesh-Aware Tokenization for Structured Graph Transformers: Learning Physical Simulation on Higher-Order Meshes

We introduce a deterministic mesh-aware tokenization strategy for graph-transformer-based physical simulation on higher-order meshes. Instead of learning token assignments, the proposed method constructs structured tokens directly from the corner nodes of higher-order finite elements and couples them to the fine mesh through parameter-free aggregation and interpolation operators derived from finite-element shape functions. This provides a discretization-aligned interface for global transformer attention without converting higher-order meshes into first-order graphs. Building on this formulation, we propose the Interpolation Mesh Graph Token Transformer (IMGTT), which combines local MeshGraphNet-style message passing with iteratively coupled global attention over mesh-derived tokens. Unlike the dual-graph MS-IMGN architecture, IMGTT replaces coarse message passing with transformer processing while preserving direct higher-order compatibility. Across static and dynamic benchmarks and element orders one through five, IMGTT consistently improves long-horizon rollout behavior over learned-token MGN-T while retaining competitive predictive accuracy across all evaluated domains. Compared with the native higher-order MS-IMGN baseline, IMGTT reduces processor parameter count by 72% and achieves approximately 3x faster training and inference. These results demonstrate that structured mesh-derived tokenization provides an effective interface between higher-order graph simulation and global transformer processing.