Learning to Read Protein Language Models: Input-Conditioned Fusion of Layerwise Representations

Protein language models (PLMs) learn structure- and function-related representations through pretraining, and recent advances in structure-aware and multimodal modeling have further enriched their representational capabilities. However, existing research primarily focuses on improving representations through pretraining or encoder enhancement, while downstream applications typically rely on the final layer or fixed layer readout strategies, with limited systematic investigation into how to fully exploit existing multilayer information without modifying the pretrained encoder. Through layerwise evaluation, we find that the final layer is not the optimal source of representations for multiple downstream tasks, and that effective readout locations vary across models and tasks, indicating the need to distinguish pretrained models' representational capabilities from their downstream readout strategies. Based on this observation, we propose a protein-specific adaptive layer fusion method, which learns a mapping from multilayer protein representations to layer weights using supervision from the target task, producing corresponding fused representations for different inputs while keeping the pretrained encoder frozen. Our evaluation across multiple representative PLMs, three datasets and four tasks shows that reusing multilayer representations can improve over final-layer readout, and when applied to base PLMs, our method can outperform models improved through encoder enhancement. As our findings indicate, the downstream capabilities of PLMs depend not only on what representations are learned, but also on how these representations are read out for the target task, with representation learning and representation readout constituting two interrelated research dimensions.