acceptodds
Under review as a conference paper at ICLR 2027

Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment

Abstract

Pretrained biological language models encode rich probabilistic structure through native token distributions, but multimodal adaptation typically aligns hidden representations or replaces the native likelihood interface with task-specific heads. We introduce LogiCA (Logit-space Contrastive Alignment), whose central methodological change is to align the context-conditioned native token distributions themselves rather than pooled latent representations. Using native-head-preserving cross-modal adapters, LogiCA couples models with distinct vocabularies and tokenization schemes without requiring a shared tokenizer, decoder, or embedding space. Contrastive learning is performed directly through each model's original token-likelihood interface, retaining token-level scoring, interpretation, and generation. Across protein–ligand, TCR–peptide, and drug-resistance tasks, LogiCA outperforms matched latent-contrastive and conditional-MLM baselines, particularly in low-homology, mutation-local, and data-scarce regimes. These results support native-logit alignment as a general strategy for making pretrained biological token distributions context-sensitive without sacrificing their probabilistic semantics.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Biological language models, protein language models, logit-space alignment, contrastive learning, masked language modeling, context-conditioned prediction, variant effect prediction, mutation ranking, protein–ligand interactions, TCR–peptide recognition, drug resistance prediction, multimodal learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gN1oOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes how to fine tune biological language models from different modalities to create representations for interacting systems. The paper connects pretrained models with cross attention adapters, and uses the conditional log likelihood to apply a contrastive objective. They benchmark this on binding prediction, drug resistance, and mutation effect scoring, and show modest to significant iimprovement across benchmarks.

Critical strengths
  1. The method is reasonable, clearly defined, and seems to work. For a methodological contribution this is quite important.
  2. While I've seen individual components of this before (log likelihoods for contrastive fine-tuning), I haven't seen it specifically applied to multimodal/interacting systems.
Critical weaknesses

When evaluating a paper like this, I'm mostly considering whether I think the methodology is something the ai x bio community will find interesting. I think this can occur in two main ways: a clever method that we didn't think of, or very strong results that will inform future work.

My main issue is I don't see that this work satisfies either of these approaches. There isn't anything very new in the method; the primary advance there seems to be applying contrastive objectives to different modalities. I think this is a natural thing to do, but somewhat unsurprising. Turning to the results, they seem to be fairly modest. Given these points, I have a hard time understanding what methodological value this would provide to the machine learning community. I qualify this by noting I'm not an expert on any of the three tasks the authors use, so I invite discussion on this point.

I think if the authors could demonstrate that this approach is useful for a more substantive task (e.g., binder design), that would be compelling. There are databases like STRING for protein-protein interactions where the contrastive objective might be useful, though converting it to a generative approach seems challenging and potentially beyond this work. The authors do explicitly note that this approach could be used for generation, but this doesn't seem to be present in the main text.

Questions for authors
  1. Can this be used for generation? if the generative results are significant, I would change my score to an acceptance.
6 comments
AuthorsOpenReview ↗

R1. New quantitative evidence of context-conditioned sequence generation

We thank the reviewer for this constructive and decisive feedback. We agree that while iterative Gibbs sampling was noted as an inherent capability of LogiCA in the initial submission, its empirical validation was not demonstrated adequatly. In response, we have carried out a comprehensive set of generative experiments across both protein–ligand and TCR–peptide systems, demonstrating that context-conditioned sequence design emerges directly from LogiCA's preserved logits without auxiliary decoders or generative training.

1. Benchmark design and generalization regimes

Crucially, Gibbs sampling in LogiCA performs iterative multi-step sequence design, not merely single-point mutation ranking. Starting from a query sequence, LogiCA samples substitutions from partner-conditioned token distributions and accepts updates guided by local likelihood scores, navigating combinatorial fitness landscapes directly.

To benchmark this systematically beyond our original oncology resistance screens, we curated 103 protein–ligand deep mutational scanning (DMS) datasets totaling 530,565 measured variants across three evaluation regimes:

  • Zero-shot: Evaluated using the BindingDB-pretrained checkpoint without any DMS supervision.
  • LOCO-30% (Leave-one-cluster-out): Fine-tuned while holding out entire 30% sequence-homology clusters.
  • LOLO (Leave-one-ligand-out): Fine-tuned while holding out unseen chemical scaffolds.

2. Generative enrichment results

Generated sequences were evaluated retrospectively against experimental fitness landscapes, measuring top-10% enrichment over random sampling:

Generative Benchmark (12-step Gibbs)LogiCALogiMLMRelevant Baseline
Protein–ligand, Zero-shot1.53×—ESM-2: 1.43×
Protein–ligand, LOCO-30%1.85×1.55×SiteFuse: 1.17×
Protein–ligand, LOLO2.12×1.52×ESM-2: 1.39×
TCR–peptide (Full chain mutations)4.51×1.42×TULIP-TCR: 1.40×
TCR CDR3 mutations1.65×0.89×TULIP-TCR: 1.31×

Values report top-10% enrichment relative to random baseline (1.0×). On LOCO-30%, LogiCA's margin over LogiMLM is statistically significant under gene-collapsed paired testing (p=1.55×10−3p = 1.55 \times 10^{-3}).

3. Multi-mutant rescue on combinatorial landscapes (TtgR)

To test whether LogiCA can navigate combinatorial fitness landscapes rather than merely prioritize individual substitutions, we evaluated 15-step Gibbs trajectories on TtgR (15,216 experimentally characterized variants, up to four simultaneous mutations). Starting from deleterious multi-mutants, LogiCA reaches +1.289σ experimental fitness and 1.92× top-10% enrichment, compared with +0.674σ / 1.56× for ESM-2 and +0.184σ / 1.15× for SiteFuse.

Together, these results confirm that LogiCA’s logit preservation provides a faithful, partner-conditioned energy surface capable of directed combinatorial sequence optimization, fulfilling the generative promise of the architecture without task-specific retraining.

We have now integrated all these new results into revised Sections 2.5 and 3.1–3.2, Tables 2–3, and expanded Appendix Sections D–E. accordingly

AuthorsOpenReview ↗

R2. Methodological novelty and significance for AI × Biology

"I'm mostly considering whether I think the methodology is something the ai x bio community will find interesting. I think this can occur in two main ways: a clever method that we didn't think of, or very strong results that will inform future work."

We really appreciate this framing. Evaluating methodological significance through either a new conceptual angle or results strong enough to steer future work is exactly what motivated LogiCA. We believe our revised work provides evidence for both.

1. Conceptual novelty: Aligning native logits rather than pooled representations

The core idea isn't just "applying contrastive learning to biology." The bottleneck we wanted to tackle is where and how biological models interact. Standard CLIP-style models align pooled representations to produce scalar compatibility scores. While effective for retrieval, these scores no longer retain the native token-probability interface. Once interaction is reduced to scalar similarity, we cannot directly ask: "What residue should go here in the presence of this molecule?"

This distinction has practical consequences. On LOCO-30%, pooled latent methods such as Boltz-2, DrugCLIP, and DrugBAN remain near random (ρ≤0.042\rho\leq0.042) under their evaluated adaptation protocols. LogiCA instead aligns models through context-conditioned native token log-likelihoods, allowing a biological partner to reshape the original token distributions:

  • Native vocabularies and probabilities remain intact, enabling heterogeneous BioFMs to interact without a shared decoder or tokenizer.
  • Mutation scoring, Gibbs generation, and cross-modal dependency maps emerge from the same model, without separate task-specific heads or generative decoders.
  • Contrastive supervision directly optimizes token likelihoods, discriminating subtle local changes while preserving the pretrained probabilistic interface.

This is not simply conditional MLM with a different loss. Matched LogiMLM and latent-contrastive controls isolate the contributions of contrastive supervision and native-logit scoring, showing that both matter, especially under scarce supervision.

2. Empirical results that guide future modeling

What excites us most is seeing this distinction translate into substantial improvements across biological settings, revealing when native-logit alignment matters most.

  • Out-of-distribution clusters (LOCO-30%): LogiCA reaches 4.06×4.06\times top-1% enrichment, versus SiteFuse (2.62×2.62\times) and LogiMLM (2.58×2.58\times), while leading in correlation (ρ=0.313\rho=0.313). With only 1% of held-in DMS supervision, LogiCA already exceeds LogiMLM trained on 100%.
  • Unseen chemical scaffolds (LOLO): LogiCA achieves 4.62×4.62\times enrichment, versus SiteFuse (2.75×2.75\times) and LogiMLM (2.90×2.90\times), demonstrating transfer beyond ligand scaffolds seen during adaptation.
  • Clinical stress test (Oncology LOO): Where unconditional sequence models and evaluated latent baselines remain below random enrichment (<0.75×<0.75\times), LogiCA achieves 1.42×1.42\times, demonstrating the value of drug-conditioned mutation scoring.
  • Native sequence generation: Without an additional decoder, LogiCA supports iterative Gibbs design, reaching 1.85×1.85\times enrichment on LOCO-30% and 2.12×2.12\times on LOLO, versus SiteFuse (1.17×1.17\times/1.19×1.19\times) and LogiMLM (1.55×1.55\times/1.52×1.52\times). Beyond single mutations, on the TtgR combinatorial landscape, 15-step generation reaches +1.289σ+1.289\sigma measured fitness, versus +0.674σ+0.674\sigma for ESM-2 and +0.184σ+0.184\sigma for SiteFuse, without generative training.

These findings reveal an important pattern: although positive-only MLM can recover much of the global fitness landscape with sufficient data, contrastive alignment retains a substantial advantage in prioritizing rare, high-functioning variants. Moreover, the same token distributions support ranking, structural interpretation, and iterative design without separately trained downstream heads.


The broader takeaway we hope the AI ×\times Bio community takes from this work is simple: what we choose to align matters as much as how we align it. Aligning native token likelihoods rather than pooled representations provides a richer probabilistic interface connecting multimodal discrimination to mutation-level interpretation and sequence design. When the goal is not merely to classify interacting pairs, but to engineer biological sequences under specific molecular contexts, preserving and contrastively aligning native logits offers a distinct and practically valuable modeling strategy.

AuthorsOpenReview ↗

R3. Broader applications and future impact of LogiCA

There are databases like STRING for protein-protein interactions where the contrastive objective might be useful. Yes! Applying LogiCA to large-scale interactomes like STRING-DB is a natural and exciting direction, and extending our framework to protein–protein interactions (PPI) is very much on our active roadmap. Still, our deliberate priority in this submission was to first tackle what we view as the hardest conceptual testbed: coupling foundation models that speak entirely different biological languages.

Pairings with disjoint vocabularies and distinct tokenizers—such as ESM-2 with SELFormer (protein–chemical) and TCRLang with ESM-2 (immune repertoire–antigen)—are precisely where standard multimodal alignment fails most visibly. In those settings, collapsing representations into pooled scalar compatibility scores completely strips away token-level distributions, making conditional sequence generation impossible. Demonstrating that native logits can be preserved and directly aligned across fundamentally heterogeneous representations was the critical methodological bottleneck we set out to resolve.

By contrast, PPI operates in an ESM–ESM regime where both interacting partners share the same amino-acid vocabulary. While conceptually more homogeneous than cross-modal pairings, this setting unlocks distinct advantages that make it a compelling next step:

  • Bidirectional interface co-design: Because both partners expose native amino acid logits, Gibbs sampling can operate symmetrically—mutations sampled on protein AA dynamically update the likelihood surface of binding partner BB, enabling joint, iterative optimization of complex interfaces.
  • Complex-conditioned DMS ranking: It provides a path to predict mutation effects on binding affinity across the interactome directly from sequence pairs, bypassing the need for resolved 3D structures.

More broadly, this highlights what excites us most about LogiCA: it offers a general, modular recipe for converting standalone masked language models into context-conditioned design engines. The exact same formulation applies whenever a sequence must be optimized against external biological constraints:

  • Single-cell & Perturbation Biology: Coupling masked gene models (e.g., GenePT, scGPT) with chemical encoders to model drug-induced transcriptional shifts and predict perturbation responses at single-gene resolution.
  • RNA–Protein Recognition: Conditioning RNA language models on RNA-binding proteins to design synthetic untranslated regions (UTRs), guide RNAs, or translation-regulating motifs.
  • Antibody–Antigen Affinity Maturation: Contextualizing antibody CDR loops directly against target antigen sequences for guided in-silico optimization.

We share the reviewer's vision for this broader impact. Aligning native token probabilities rather than discarding them for scalar scores transforms pretrained biological foundation models into generative, context-aware design systems, and we are eager to see how the community builds on this foundation.

Reviewer_YyQrOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper proposed to align the context-conditioned token distributions rather than the pooled latent representations so that it can leverage models pre-trained with different vocabularies and token schemes. The paper demonstrated performance gain on protein-ligand, TCR-peptide, and drug-resistance tasks in comparison to controls (wo token-likelihood scoring and wo negatie pairs).

Critical strengths

One benefit of the work is that it identifies the tasks for which scoring the mutation sites works better than directly comparing the pooled latent features. For instance, in Table 1, it seems that the proposed methods that require prior knowledge of A and the pocket-based method perform the best.

Critical weaknesses

My biggest concern is on the requirement of AA, which denotes the positions at which the score is evaluated. For sequence-level matching, they define AA as the full sequence of length ∣L∣|L|. This means that a pass is required for each position in AA, so the computation seems quite heavy. Also it is difficult to judge whether iterating over every position is necessary, as I cannot find experimental results to support this. For localized tasks, AA is defined as mutation sites, binding-interface residues, or other biologically meaningful subsets. However, it is unclear how costly it is to annotate AA. Additionally, the relationship between performance and the percentage of pairs for which$A$ is unknown during both inference and testing is unclear to me.

I don't fully understand the purpose and/or the contribution of proposition 1 as the current version is a simple algebra equation based on the matching mutation sets. I believe the subtraction only matters when the mutation set does not match, so understanding the effect across variants with different mutation sets would be helpful. Additionally, as the score is averaged over M, when the two variants have different mutation sets, what would the relationship be?

Questions for authors

What is the reason for not including Boltz-2 in Table 1? Additionally, Boltz-2 seems to perform relatively bad in Table 2, any clarification on that would be helpful.

8 comments
AuthorsOpenReview ↗

R1 — Computational cost: exact masking versus single-pass approximation

This means that a pass is required for each position in A, so the computation seems quite heavy.

This is a crucial point of clarification, and we thank the reviewer for raising it. Our original manuscript did not clearly separate the formal masked-likelihood definition from its actual inference implementation. In practice, LogiCA does not run LL separate forward passes. Instead, following standard practice in protein language modeling (e.g., ESM-1v wild-type marginal scoring), LogiCA computes contextualized token distributions across all positions simultaneously in a single forward pass over the wild-type sequence. To address computational overhead definitively, we benchmarked wall-clock inference latency on an NVIDIA H100 GPU and evaluated whether this single-pass approximation sacrifices ranking accuracy:

Inference Method (NVIDIA H100)Latency / pairSpearman ρ\rho (LOCO-30%)
LatentCA (pooled contrastive baseline)4.42 ms0.089
LogiCA (single-pass WT-marginal)4.55 ms0.313
Exact LL-pass position-masked scoring2,047.5 ms0.312

Two key takeaways resolve the computational concern:

  1. Minimal per-pair overhead relative to contextualized latent scoring (~2.9%): LogiCA's single-pass wild-type-marginal approximation requires 4.55 ms versus 4.42 ms for the matched contextualized latent baseline. Importantly, this comparison concerns scoring an individual sequence–context pair, not exhaustive library retrieval, where cached dual encoders remain substantially faster. LogiCA is therefore particularly suited to mutation-library scoring and candidate reranking.
  2. ~450× speedup with virtually unchanged ranking accuracy: Compared with exact position-masked scoring requiring one pass per residue, the single-pass wild-type-marginal approximation reduces latency from 2,047.5 ms to 4.55 ms, while achieving nearly identical mutation-ranking correlations (ρ=0.313\rho=0.313 vs. 0.3120.312). This demonstrates that repeated masking is unnecessary for the evaluated single-substitution ranking task.

Evaluating an entire deep mutational scanning library (thousands of single-site variants) requires only one forward pass for the wild-type target paired with the ligand, after which all variant scores are extracted via parallel matrix indexing.

We have revised Section 2.3 and Appendix C to explicitly detail the single-pass wild-type marginal algorithm and incorporate this latency-accuracy benchmark.

AuthorsOpenReview ↗

R2. Clarification of the scoring set A and annotation requirements

R2. Clarification of the scoring set A and annotation requirements

For localized tasks, A is defined as mutation sites, binding-interface residues, or other biologically meaningful subsets. However, it is unclear how costly it is to annotate.

We understand the question to be whether LogiCA requires prior knowledge of functionally important positions to achieve its reported performance. Please let us know if we have misunderstood the question. Importantly, none of our main experiments requires such annotations. For sequence-level protein–ligand binding, we use A=[L]A=[L], meaning all sequence positions contribute to the score. For mutation-effect prediction, A=M(x,xwt)A=\mathcal{M}(x,x^{\mathrm{wt}}) simply identifies positions where a candidate differs from its wild-type reference.

  • Deep mutational scanning (DMS) datasets experimentally measure the effects of large numbers of substitutions. LogiCA predicts these effects directly from the sequences and their biological context, without using the corresponding experimental measurements or knowing which positions are functionally important. The measured activities or affinities serve as ground truth for evaluating predictions (and as training labels only for held-in datasets during LOO adaptation).

  • Similarly, binding-site and structural-contact annotations are used only for retrospective evaluation of LogiCA's zero-shot dependency maps, never as inputs to those predictions.

Thus, the scoring set A introduces no additional functional-annotation cost, and our reported performance does not depend on prior knowledge of functional or binding residues. We hope this addresses the intended concern, but would be happy to clarify further if the reviewer meant something different.

AuthorsOpenReview ↗

R3. Proposition 1 and different mutation sets

"I believe the subtraction only matters when the mutation set does not match, so understanding the effect across variants with different mutation sets would be helpful."

Proposition 1 was indeed designed as a clean algebraic illustration of how preserving native token likelihoods cancels shared wild-type reference contributions. Crucially, this exact cancellation generalizes to variants with different or non-overlapping mutation sets, provided both variants are evaluated on a common support. We have now extended the theoretical analysis to make this generalization explicit.

For any two variants AA and BB with mutation sets MAM_A and MBM_B, define their union support U=MA∪MBU=M_A\cup M_B. Evaluating both variants relative to the same wild-type reference over UU yields exact cancellation of the shared reference terms:

ΔU=1∣U∣∑i∈U[log⁡πθ(xiA∣x∖iA,y)−log⁡πθ(xiB∣x∖iB,y)].\Delta_U = \frac{1}{|U|}\sum_{i\in U}\left[\log\pi_\theta(x_i^A\mid x_{\setminus i}^A,y)-\log\pi_\theta(x_i^B\mid x_{\setminus i}^B,y)\right].

This common-support formulation provides two important properties:

  1. Principled common-support normalization: Both variants are compared over the same positions and denominator ∣U∣|U|, including when they carry different numbers of mutations (e.g., single versus double or triple mutants).
  2. Exact relative scoring with contextual effects: The wild-type reference terms cancel exactly, without assuming that conditional likelihoods remain unchanged at positions mutated in only one variant. Consequently, contextual and epistatic effects are retained.

Importantly, we also distinguish this common-support comparison from the individually normalized mutation scores used for variant ranking in practice. These are not generally identical when mutation sets differ. Our revised analysis derives their exact relationship:

∣U∣ΔU=∣MA∣sMA(A)−∣MB∣sMB(B)+RAB,|U|\Delta_U=|M_A|s_{M_A}(A)-|M_B|s_{M_B}(B)+R_{AB},

where RABR_{AB} captures changes in the conditional likelihoods of wild-type tokens at positions mutated in only one candidate. This residual vanishes under an explicit context-invariance condition, but need not vanish in general.

Thus, the contribution is not the algebraic cancellation itself, but the ability to express variant comparisons directly through context-conditioned, position-level likelihoods while explicitly characterizing the effects of different mutation supports and surrounding sequence context. Pooled latent similarities do not generally provide this decomposition.

We have expanded the discussion following Proposition 1 in Section 2.2 and the theoretical appendix to establish the common-support cancellation, derive the residual correction, and clarify its relationship to the practical variant-ranking score.

Reviewer_NkUeOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces LogiCA, a framework for aligning pretrained biological language models across different modalities through contrastive learning on native token likelihoods rather than pooled latent representations. The approach uses cross-modal adapters to exchange information between models with different vocabularies while preserving their original token prediction heads. The authors evaluate LogiCA on protein–ligand binding prediction, drug-resistance variant ranking, and TCR–peptide recognition using ESM-2, SELFormer, and TCRLang. On a low-homology BindingDB benchmark, LogiCA achieves an AUROC of 0.630, outperforming matched latent-contrastive (0.604) and conditional MLM (0.547) baselines. The approach also improves drug-conditioned mutation ranking and zero-shot TCR–peptide variant ranking, particularly on peptide mutations. Additional experiments examine mutation-local scoring, cross-modal dependencies, backbone scaling, and data efficiency.

Critical strengths
  • The paper introduces an interesting approach to multimodal alignment by applying contrastive learning directly to contextualized token likelihoods rather than pooled representations. Preserving native token prediction heads is well motivated for biological applications involving mutation scoring and sequence generation.
  • The experimental evaluation covers diverse biological tasks, including protein–ligand binding, drug resistance, and TCR–peptide recognition, demonstrating the potential applicability of the proposed approach across different biological modalities.
  • The authors provide well-controlled comparisons with latent contrastive learning and conditional MLM using matched backbones and training protocols. These comparisons help isolate the benefits of the proposed scoring and alignment objective.
  • The paper includes detailed analyses of low-homology generalization, mutation-local scoring, data efficiency, and cross-modal dependencies. The results under limited paired supervision are particularly interesting.
Critical weaknesses
  • The TCR–peptide results show considerable variation across evaluation settings. In particular, the ATLAS-TCR benchmark contains only seven mutation groups with large standard deviations. Moreover, the TCR-anchored and peptide-anchored models use different numbers of negative samples during pretraining (1 vs. 5), making it difficult to determine whether the observed performance differences are primarily caused by the anchoring direction or the training configuration.
  • The TCR–peptide pretraining relies on synthetic negatives generated through single-residue mutations of known binding pairs. However, these mutations do not necessarily eliminate binding activity. Although the authors provide evidence that most mutations do not improve activity over wild type, this does not establish that the resulting pairs are nonbinding. Further analysis of how negative construction affects training and downstream performance would strengthen the approach.
  • The paper emphasizes that preserving native token likelihoods enables generation and interpretation, but the validation of these broader capabilities is somewhat limited. While the authors evaluate cross-modal dependency maps, generation is primarily described through a Gibbs sampling procedure without quantitative evaluation of generated sequences. In addition, the computational overhead of pair-specific contextualization is not directly compared with the latent baselines at inference time.
Questions for authors
  • In Table 2, some improvements over LogiMLM are relatively small but reported as statistically significant. Could the authors clarify how the paired t-tests were conducted? In particular, were gene–drug assays treated as independent observations, or was the dependence between assays from the same gene considered?
  • The TCR-anchored and peptide-anchored LogiCA models use different numbers of negative samples during pretraining. How much of the performance difference between these models is attributable to the anchoring direction rather than the negative sampling configuration? Would the same trend hold with matched negative sampling budgets?
  • The TCR–peptide pretraining uses single-residue mutations as negative examples. However, Figure S3 reports the fraction of mutations that improve activity over wild type, which does not necessarily establish loss of binding. Could the authors clarify how this observation supports the negative construction strategy and whether the results are robust to alternative negative sampling methods?
6 comments
AuthorsOpenReview ↗

R1: Statistical significance and dependence between gene–drug assays

In Table 2, some improvements over LogiMLM are relatively small but reported as statistically significant.

Yes, the original analysis used paired t-tests across 90 Coelho gene–drug assays and 10 EGFR drug assays, comparing both methods on the same experimental variants within each assay. Thus, significance reflects the consistency of the paired improvements, rather than simply the magnitude of the difference between average scores. Even small improvements can be significant when evaluated across sufficiently many assays, particularly when the methods have strongly correlated performance, as is the case here.

Nevertheless, we recognize that treating gene–drug assays as independent does not fully account for shared-gene dependence, and that statistical significance alone does not establish a substantial practical advantage. We have therefore strengthened our evaluation in three ways.

1. Larger and more diverse experimental validation. We expanded the evaluation beyond oncology drug resistance to 103 protein–ligand DMS cohorts (530,565 variants) across*42 genes and 32 homology-disjoint clusters, including direct biochemical and ligand-dependent functional measurements. Crucially, the improvement remains significant after accounting for shared-gene dependence through gene-level aggregation (Δρ = +0.0234, p = 6.98 × 10⁻⁴) and homology-cluster aggregation (p = 3.30 × 10⁻⁴).

2. Stronger gains in identifying high-functioning variants. We now additionally report top-1% enrichment, a metric more directly relevant to experimental discovery, where the objective is often to identify a small number of high-performing variants rather than accurately rank the entire fitness landscape. On LOCO-30%, LogiCA achieves 4.06× enrichment versus 2.58× for LogiMLM, a 57% relative improvement, despite a more modest difference in Spearman correlation (0.313 versus 0.293). This advantage is consistent with LogiCA's contrastive objective, which explicitly compares high-functioning variants against hard negatives, whereas LogiMLM optimizes token reconstruction without directly emphasizing the highest-performing candidates.

3. Larger improvements under limited supervision. We further investigated how the advantage depends on the amount of available training data. Specifically, we varied the fraction of held-in DMS supervision used for fine-tuning, while keeping the evaluation on entirely unseen protein clusters (LOCO-30%, with no target-cohort labels).

Training dataSpearman ρ LogiCA / LogiMLMTop-1% enrichment LogiCA / LogiMLM
1%0.310 / 0.2363.47× / 2.10×
2%0.312 / 0.2513.60× / 2.48×
5%0.312 / 0.2773.70× / 2.59×
10%0.313 / 0.2844.02× / 2.67×
25%0.314 / 0.2894.04× / 2.61×
50%0.315 / 0.2914.02× / 2.60×
100%0.313 / 0.2934.06× / 2.58×

The results reveal two important trends. First, the Spearman advantage is substantially larger when supervision is scarce: at 1% training data, the improvement is Δρ = +0.074, compared with +0.020 at full supervision. Remarkably, LogiCA trained with only 1% of the data already outperforms LogiMLM trained with 100%. Second, the advantage in top-1% prioritization remains substantial even as more data become available, indicating that contrastive alignment is particularly effective at identifying rare high-functioning variants.

We have incorporated these experiments into Section 3.1 and Table 2.

AuthorsOpenReview ↗

R2. TCR benchmark variability and matched negative-sampling budgets

The TCR–peptide results show considerable variation across evaluation settings. The ATLAS-TCR benchmark contains only seven mutation groups with large standard deviations. Moreover, the TCR-anchored and peptide-anchored models use different numbers of negative samples during pretraining (1 vs. 5), making it difficult to determine whether the observed performance differences are primarily caused by the anchoring direction or the training configuration.

We thank you for highlighting these concerns.


1. Biological variability and expanded evaluation. First, we would like to clarify that substantial variability across TCR–peptide mutation landscapes is biologically expected. Different receptor–epitope pairs exhibit markedly different recognition specificities and mutational tolerances, making some landscapes considerably harder to predict than others. Importantly, the reported dispersion represents standard deviations across independent DMS groups, not standard errors of the mean. Thus, large standard deviations reflect the heterogeneity of the biological systems being evaluated and do not, by themselves, imply that the mean improvement is statistically unreliable.

However, we agree that the original ATLAS-TCR benchmark, containing only seven mutation groups, was too small to support our conclusions about TCR-side mutation ranking or anchoring direction. To address this, we expanded TCR-mutation evaluation to 23 DMS groups (703 variants across 18 index TCRs), incorporating TRAIT, SKEMPI v2, and two 868-Z11 CDR3α/β saturation scans. Importantly, this expanded evaluation led us to revise our original conclusion. Whereas the smaller ATLAS-TCR benchmark suggested that peptide anchoring was preferable for TCR mutations, the expanded results show that TCR-anchored LogiCA achieves the strongest overall mutation-ranking performance on both modalities:

ModelPeptide DMS Spearman ρTCR DMS Spearman ρ
LogiMLM0.0600.044
LogiCA-TCR0.2710.190
LogiCA-Pep0.1790.105
LogiCA-Dual0.2230.059

LogiCA-TCR also achieves the highest top-10% prioritization enrichment on both peptide (2.62×) and TCR (1.60×) mutations. Its improvements over LogiMLM are statistically significant (p = 5.2 × 10⁻⁹ and p = 0.0087, respectively). Furthermore, epitope-clustered permutation testing on BATCAVE-Full confirms that the improvement remains significant after accounting for shared epitopes (Δρ = +0.229, p = 0.0023).


2. Controlling for negative-sampling budget. We also agree that comparing models trained with K=1 and K=5 negatives confounded anchoring direction with training configuration. To disentangle these effects, we performed the requested controlled 2 × 2 factorial experiment, evaluating both anchoring directions at matched K=1 and K=5:

AnchorKPeptide DMS ρTCR DMS ρ
TCR10.2710.190
Peptide10.249−0.011
TCR50.1790.156
Peptide50.1790.105

These experiments demonstrate that the advantage of TCR anchoring is not simply explained by the number of negatives. At matched K, TCR anchoring matches or exceeds peptide anchoring on the expanded peptide benchmark and performs better on TCR-mutation ranking. A compelling explanation is the strong asymmetry of the pretraining repertoire (~166k unique TCRs versus ~1.6k peptides), which makes contrasting local peptide substitutions against a fixed TCR a particularly informative learning signal. Nevertheless, peptide anchoring remained advantageous for some individual benchmarks and iterative Gibbs generation, indicating that the optimal configuration also depends on the downstream objective.


Overall, your concern prompted an important revision of our interpretation, and we sincerely thank you for bringing this to our attention. Rather than concluding that each anchoring direction specializes in mutations of the opposite modality, our expanded evaluation now identifies TCR anchoring as the strongest general-purpose configuration for mutation ranking across both modalities, while peptide anchoring retains complementary advantages for generative design.

We have incorporated these additional experiments and analyses into Section 3.2 and Table 3, with further details on the controlled anchor-direction ablations, expanded TCR benchmarks, and epitope-clustered significance analysis provided in the Appendix.

AuthorsOpenReview ↗

R3. Validity of synthetic mutation negatives and alternative strategies

The TCR–peptide pretraining relies on synthetic negatives generated through single-residue mutations of known binding pairs. However, these mutations do not necessarily eliminate binding activity. Although the authors provide evidence that most mutations do not improve activity over wild type, this does not establish that the resulting pairs are nonbinding. Further analysis of how negative construction affects training and downstream performance would strengthen the approach.

We thank the reviewer for this important observation. We agree that reduced activity relative to wild type does not necessarily imply loss of binding. We would like to clarify that our pretraining procedure relies on a two stage pipeline which was not sufficiently explained in our original submission. Synthetic negatives were introduced to address the scarcity of experimentally annotated non-binding TCR–peptide pairs, but they are used only during the first stage.

1. Pretraining with synthetic mutations. During the first stage, we use experimentally supported TCR–peptide binding pairs and generate local single-residue mutations as contrastive alternatives. The objective is not to establish that every mutation abolishes binding, but rather to encourage the model to distinguish an observed binding pair from closely related sequence perturbations. Such local contrasts are particularly important because single-residue mutations produce highly similar representations in the pretrained BioFM despite potentially substantial changes in recognition.

2. Supervised fine-tuning with observed binding pairs. In the second stage, we no longer use synthetic point mutations as negatives. Instead, we fine-tune on experimentally supported TCR–peptide binding annotations, constructing negative pairs by shuffling experimentally observed peptides that are not annotated as binders of the corresponding TCR. Thus, synthetic mutations are used for mutation-sensitive pretraining, rather than being treated as experimentally confirmed non-binding labels during supervised fine-tuning. The downstream DMS mutation-activity measurements are reserved for evaluation.

We have clarified the two-stage training procedure and incorporated these additional analyses into Section 3.2 and the Appendix.


3. Why is the first stage important? In our initial experiments, positive-only training produced substantially weaker mutation-sensitive predictions, motivating the introduction of synthetic-negative contrastive pretraining. While we had omitted those early diagnostic runs from the initial submission for brevity, we now include the full ablation explicitly in the revised manuscript to quantify Stage 1's contribution. We compare skipping Stage 1 directly against positive-only LogiMLM and our complete two-stage LogiCA-TCR:

Evaluation metricLogiMLMFine-tuning only (supervised head)Fine-tuning only (native logits)LogiCA-TCR (two-stage)
ePytope binary AUROC0.5240.5670.5230.672
ePytope binary AUPR0.3410.3110.3300.485
Peptide mutation Spearman ρ (71 DMS)0.0600.0750.1150.271
TCR mutation Spearman ρ (23 DMS)0.0440.0890.1070.190

These results highlight an important distinction. Supervised fine-tuning alone learns broad TCR–peptide discrimination, but provides substantially weaker mutation-sensitive predictions under our evaluated training protocol. Introducing synthetic-mutation contrastive pretraining improves both binding discrimination and mutation ranking, with peptide Spearman correlation increasing from 0.115 to 0.271 and TCR correlation from 0.107 to 0.190 under native-token scoring. Importantly, these synthetic mutations are treated as contrastive alternatives rather than experimentally verified nonbinders. While some may retain binding activity, the ablation demonstrates that this local contrastive training stage provides useful supervision for downstream mutation ranking. Evaluating alternative negative-construction strategies remains an important direction for further study.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Closing Summary of Revisions and Reviewer Discussion

We thank the three reviewers for their constructive feedback, which has substantially strengthened our manuscript. To facilitate the Area Chair's assessment, we briefly summarize the outcome of the discussion:

  • Reviewer YyQr (Weak Acceptance, upgraded from Weak Rejection): We addressed the concerns regarding computational efficiency, annotation requirements, Boltz-2 comparisons, and theoretical interpretation. The reviewer explicitly acknowledged these clarifications and indicated a move toward acceptance.

  • Reviewer NkUe (Weak Acceptance): We addressed the experimental concerns through expanded benchmarks (103 protein–ligand DMS cohorts, 23 TCR-mutation groups), dependence-aware statistical tests, controlled training ablations, and quantitative evaluations of sequence optimization, structural interpretation, and inference efficiency.

  • Reviewer gN1o (Weak Rejection): The reviewer acknowledged that the additional experiments were compelling and that they establish the method's effectiveness. The remaining concern is primarily about methodological significance. We further strengthened the manuscript's motivation and demonstrated substantial gains in partner-conditioned sequence optimization (4.51× vs. 1.40× enrichment in TCR–peptide design; +1.289σ vs. +0.674σ on combinatorial TtgR optimization).

Overall, the revised manuscript addresses the major technical and experimental concerns and provides substantially stronger evidence for LogiCA's central contribution: a unified native-logit alignment framework supporting mutation ranking, partner-conditioned sequence optimization, and structural interpretation without additional task-specific decoders. We sincerely appreciate the reviewers' engagement and thank the Area Chair for considering the revised manuscript and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.