Shared Constraints Dominate Protein Language Model Generalization

When foundation models are trained on data pooled across heterogeneous generative contexts, successful zero-shot transfer can occur via two distinct mechanisms. The model may dynamically condition on latent context-specific structure, or it may rely primarily on regularities invariant across the training distribution. Protein language models (PLMs) provide an opportune testbed for this distinction, as they are trained on diverse evolutionary corpora and yet evaluated on zero-shot mutation-effect prediction defined by individual, context-dependent fitness landscapes. In this paper, using a controlled synthetic evolutionary framework with exact oracles and then a natural-corpora setting, we investigate whether PLMs learn task-invariant or task-specific structure. In both settings we find that PLM predictions track mutation effects common across training contexts rather than specific to a target. Intervening on training diversity reveals that broader ensembles strengthen this shared signal, while suppressing shared structure causes prediction to collapse rather than triggering a switch to task-specific structure. The synthetic setting reveals that learned representations encode fine-grained information about which lineage a sequence belongs to. Similarly, in the natural setting, where lineage is operationalized as a cluster of similar homologs, cluster identity is indeed decodable from local hidden states. However, we find that models of various scales do not employ this information to predict mutation effects. Altogether, our findings suggest that PLM generalization on mutation tasks is fundamentally driven by shared constraints, providing new insights into how sequence models may handle latent distribution shifts.