Human-specific Variation Complements Biological Language Models

Modern biological language models are trained on vast sequence corpora representing organisms across the tree of life, enabling models to learn representations of biological sequences that generalize across a wide range of evolutionarily disparate organisms. However, for deepening understanding of human biology and disease, a human-centric evolutionary timescale is important. Human mutational constraint identifies genomic regions that have only recently been subject to selection within the human lineage and may have functional consequence for human-specific traits and diseases. We show that across modalities (DNA, RNA, and proteins), state-of-the-art biological language models capture deep evolutionary conservation but are weak proxies of human constraint. Furthermore, human constraint provides complementary information to the models tested, improving their ability to identify common functional variation with effects on both organismal and molecular phenotypes, despite the models' large scale pre-training. We additionally demonstrate that state-of-the-art MSA-based and sequence-to-function models can also be improved by explicit incorporation of metrics of human constraint. These results point to an unmet need to develop methods that model biological sequences not just across deep evolutionary divergence but also at a human-focused timescale.