ProbRead: Allosteric Readout and Homeostatic Fusion for Frozen Cross-Modal Biomolecular Models
Abstract
Cross-modal biomolecular models for molecules and proteins use an encoder for each modality and a projector that maps the pooled encoder output into a shared space. For downstream tasks, these models are typically frozen and evaluated by linear probing on a single representation: the pooled encoder embedding or the projected embedding. We observe that the better representation changes across tasks. Moreover, fixed pooling applies the same task-independent rule to every input, and concatenating the two pooled embeddings yields limited gains. We propose ProbRead (Probe Readout), a lightweight readout with two components: (i) Allosteric Readout uses the projected embedding only to guide token selection over the encoder tokens, replacing fixed pooling. (ii) Homeostatic Fusion assigns complementary, input-conditioned weights to the aggregated token representation and the projected embedding in each feature dimension. Across three platforms and five groups of biomolecular tasks, ProbRead achieves the best average performance, outperforming encoder-only and projector-only probing as well as linear probing of other biomolecular foundation models. ProbRead 's output also recovers the effective rank lost by projection. These results demonstrate that adaptively combining the two representations is more effective than fixing either one.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.