MM-HAGT: Hierarchical Graph Transformers Conditioned on Incomplete Clinical Data for Whole-Slide Images

Pathological staging from gigapixel WSIs requires modeling spatially organized tissue patterns together with complementary patient context. Existing WSI models typically ignore structured clinical evidence, whereas conventional multimodal fusion is vulnerable to incomplete clinical records. We introduce MM-HAGT, a hierarchical graph Transformer that accounts for missing information and incorporates clinical context. MM-HAGT represents each WSI as a graph combining topological and semantic relationships, adaptively reweights uninformative patches, and hierarchically aggregates patch features into representations at the region and slide levels. A missing aware clinical encoder transforms variables with a fixed schema and their observation mask into a token controlled by a reliability gate, which interpolates between observed clinical evidence and a learned prior for missing information. Self attention subsequently enables the clinical token to condition localized WSI graph representations. Clinical modality dropout and auxiliary supervision further promote robustness to incomplete records and discourage excessive reliance on structured variables. This framework provides a unified approach to hierarchical WSI representation learning with partially observed clinical context for pathological staging at the patient level.