HoloRoPE: Holographic RoPE for Hierarchical Wireless Foundation Models

Wireless foundation models (WFMs) extend the pretraining paradigm of foundation models to wireless communications, learning high-quality representations from large-scale channel data that serve diverse downstream wireless tasks. The input to WFMs is typically channel state information (CSI), which has two axes that index antennas and subcarriers and sample the propagation channel in space and in frequency, respectively. Adjacent subcarriers in wireless data typically lie within the coherence bandwidth, and we exploit this property by introducing Asymmetric Hybrid-Resolution Attention (AHRA), a hierarchical attention design that better suits wireless signals. AHRA aggregates local signals through pooling, after which prevailing positional encodings assign each new token a single fixed position from the original grid. However, this assignment is limiting for wireless signals, which are sensitive to positional inductive bias. By constructing the response difference between different ways of computing the positional encoding, we decompose it into the main factors that drive the positional difference. On this basis, we propose Holographic Rotary Position Embedding (HoloRoPE), which scales regional keys with learnable gains based on the geometric and content factors. Extensive experiments on large-scale wireless datasets show that both AHRA and HoloRoPE improve performance over the baseline models.