Temporal Triangular Attention for Protein Molecular Dynamics

Protein dynamics carry critical information about protein function, yet current machine learning methods offer few ways to encode and reason over them. Molecular dynamics (MD) trajectories provide a rich view of these dynamics that is complementary to the sequence and evolutionary signals exploited by protein language models, but transforming them into effective protein representations remains an open challenge. In this work, we introduce Temporal Triangular Attention (TTA), which extends triangular attention from a single static structure to a temporally evolving sequence of structures. For each residue pair, TTA jointly selects a mediating residue and a trajectory frame, enabling pair-conditioned reasoning over when and where relevant motion occurs. We instantiate TTA in ProTTA, an Evoformer-style MD encoder that incorporates residue trajectories into pairwise structural embeddings. By pretraining ProTTA to forecast the dynamic cross-correlation of the next trajectory window, we obtain representations of coordinated motion without any functional annotations. We evaluate ProTTA on two benchmarks spanning function prediction and dynamics forecasting. For function prediction, ProTTA outperforms existing MD-based encoders and, when combined with sequence information, achieves the best result on 22 of 24 metrics against both sequence models and MD encoders. For dynamics forecasting, ProTTA outperforms the MD-only baseline and performs comparably to a sequence-conditioned model while using dynamics alone.