SheaFormer: Sheaf Diffusion as a Structured Multi-Head Operator

Sheaf Neural Networks (SNNs) enrich graph diffusion with vector-valued stalks connected by learned restriction maps, and have been extended to directed hypergraphs by encoding orientation through a complex phase. We show that sheaf diffusion admits an exact stream-major decomposition whose diagonal specialization has the additive structure of the Transformer's multi-head attention: stalk coordinates act as heads, sheaf-induced node operators take the place of softmax attention matrices, and stalk and channel maps become structured value and output maps. We introduce SheaFormer, an SNN whose multi-head structure is inherited from the sheaf itself: it places sheaf propagation inside a residual block with a complex whitening of the propagated branch and generates role-aware restriction maps from the head and the tail of each hyperedge as bilinear node–context compatibilities. Whereas the phase encoding of the Directional Sheaf Hypergraph Network (DSHN) discards orientation at zero charge, the role-aware restrictions retain it and, in a single layer, distinguish orientations of a graph that no single DSHN layer with diagonal restriction maps distinguishes for any charge. Across twelve directed hypergraph benchmarks, the gain of the residual block grows as homophily decreases, while the role-aware restrictions improve eleven of the twelve benchmarks. SheaFormer attains the highest average accuracy, 2.03 points above the strongest baseline, and the highest F1 on two real-world molecular-reaction hypergraphs.