GraphTSA: Adaptive Token Communication for Local Heterogeneity and Global Synchronization in Medical Time Series

Medical time series (MedTS) exhibit a structural duality: measurements retain local heterogeneity from distributed sources and measurement geometry, while sharing global synchronization from common physiological activity. Dense pairwise attention captures heterogeneous interactions but lacks explicit global coordination; CoTAR efficiently aggregates global information but broadcasts a shared core across tokens, limiting heterogeneous communication. We propose Graph Token Structural Attention (GraphTSA), an MLP-based module that preserves CoTAR's token-message construction but replaces shared broadcast with input-adaptive, directional structural mixing without a separate value projection. A low-dimensional query–key factorization controls affinity-logit capacity. Across seven public EEG, ECG, and human-activity-recognition datasets, GraphTSA improves upon TeCh with CoTAR on six. Rank and topology analyses reveal dataset-dependent rank sensitivity, richer heterogeneous communication, and a persistent dominant spectral mode. Despite replacing TeCh's linear token mixing with adaptive pairwise communication, GraphTSA incurs less than 1% parameter and memory overhead on PTB at its best-performing rank. Code and training scripts will be released upon acceptance.