acceptodds
Under review as a conference paper at ICLR 2027

Direct or \(\Delta\)? Understanding Two Routes to Mutant Binding Affinity Prediction

Abstract

Predicting how protein mutations alter ligand binding is crucial for understanding drug response and identifying compounds effective across variants. In modern biomolecular modeling, increasingly capable models coexist with comparatively limited mutation-specific affinity data, making the formulation of the prediction target itself an important modeling choice. Mutant affinity can be predicted either directly or through its change relative to the wild type (WT), yet it remains unclear when one route should be preferred over the other. We develop a statistical framework for comparing the two routes, Direct and Delta (), under the same input. The analysis shows when WT–mutant pairing can simplify affinity-change prediction by canceling shared variation, how this advantage depends on training size, and how interacting prediction errors affect the reconstructed mutant affinity. A linear-Gaussian analysis further characterizes conditions under which the preferred route can change with sample size. Controlled simulations reproduce the predicted finite-sample behavior, while experiments on PLATINUM and DAVIS-Complete reveal distinct real-data regimes: the -route advantage narrows as Direct prediction improves on PLATINUM, whereas sample-specific affinity-change prediction becomes increasingly informative with additional data on DAVIS-Complete. These results highlight the framework as a tool both explanatory and practical for modeling. Our code will be publicly available.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Binding affinity prediction, Mutation-effect learning, Risk analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SDqCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Protein-ligand binding affinity prediction upon mutation can be predicted in absolute terms or in relative terms (with respect to a wildtype protein). This papers compares the two formulations by deriving an MSE decomposition, a linear-Gaussian finite-sample analysis, simulations, and an empirical study on two datsets PLATINUM and DAVIS-Complete.

Critical strengths

Clear and well-motivated research question: The paper studies a practically relevant target-design choice: whether mutant affinity should be predicted directly or reconstructed from a predicted mutation effect and a WT-affinity estimate. It uses a principled theoretical framework with a mathematical characterization of the trade-offs between the two routes.

The authors combine controlled simulations with experiments on two real-world datasets, PLATINUM and DAVIS-Complete

Carefully matched comparison: Direct and Delta learners use the same inputs, architecture, training examples, seeds, and nested subset chains, substantially reducing implementation-related differences between the two routes

Critical weaknesses
  1. The novelty of the mathematical contribution is unclear. The central derivations use a variance identity for paired differences, squared-error decompositions, and expected OLS risk under a linear-Gaussian model. The paper does not clarify which of these results are intended to be novel or position them relative to the extensive literature on paired designs, change scores versus baseline-adjusted regression, control variates, and multi-fidelity or delta learning. The Related Work section is insufficient and largely restricted to molecular prediction.

  2. The practical scope of the absolute-versus-relative comparison is unclear. The paper frames this as absolute vs relative prediction; it is unclear whether this dichotomy is practically meaningful as formulated. Existing methods already combine absolute and relative supervision (MuPIPR Zhou et. al., Twin Peaks Chowdhury et. al., Boltz-2 Passaro et. al. …), and the manuscript’s may benefit from analyzing a continuum of hybrid objectives. More importantly, the experiments do not establish that choosing between the two endpoints has a robust practical consequence: the conclusions rely on a simple, shallow architecture without comparison to established methods, a single split, and evaluations that do not adequately control for protein-family or ligand similarity. Consequently, the results may characterize behavior in this particular experimental setup rather than establish a general principle for mutant binding-affinity prediction.

  3. The fixed split does not control protein or ligand similarity. The WT-ligand context split prevents the exact same WT-ligand context from appearing across splits, but it does not enforce separation by protein family, protein-sequence similarity, structural similarity, or ligand similarity. The supplement explicitly allows ligand identities to occur across partitions, and no Tanimoto-similarity threshold is reported. Evaluation examples may contain proteins closely related to training proteins or ligands identical or highly similar to training ligands.
    Although the Direct and Delta routes are evaluated on matched examples, this potential leakage does not guarantee that such similarity benefits them equally. The Direct model, Delta model, and frozen WT anchor may exploit protein-family and ligand similarity differently. Therefore, the reported comparison establishes which route performs better under this particular partition, but not whether the difference persists for unseen protein families or ligand chemotypes. These factors may also confound the training-size trends. Increasing the training budget may therefore change dataset composition as well as sample size. Multiple seeds, randomized training subsets, and bootstrap intervals remain conditional on the same partition and do not measure sensitivity to split selection. The main conclusions should therefore be tested using repeated protein-sequence-clustered and ligand-scaffold/Tanimoto-clustered splits.

  4. The empirical evidence is too narrow. No real-data crossover is observed: PLATINUM shows only a narrowing gap, while DAVIS shows no resolved route trend. The conclusions also depend on one untested architecture, one frozen-anchor construction, and one training protocol. Yet the route advantage is explicitly a function of the Direct learner, Delta learner, anchor, and their interaction. The authors make sure to have a fair compairison across the two routes in terms of architecture. However, comparing the trained model to known baselines or starting from known architectures and adapting them to this analysis would provide a more practical analysis that is not restricted to the very specific architectural decisions made on the paper. The paper does not report a constant mutant-affinity baseline so the absolute competence of the models cannot be assessed. On PLATINUM, WT plus mean Δ\Delta outperforms WT plus learned Δ\Delta; thus the route advantage does not demonstrate successful mutation-specific learning.

Questions for authors
  1. How were the 370 and 831 examples selected, what type of mutations, are they all on the interface with the ligand, etc..?
  2. Do the conclusions hold under repeated protein-sequence-cluster and ligand-scaffold/Tanimoto-clustered splits?
  3. Do route differences remain after stratifying or adjusting for protein family, ligand properties, mutation location and class?
  4. Consider a continuum experiment with some interpolated training target or weighted loss objective.
Reviewer_uaadOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper suggests the difference between M (affinity between a mutated protein and ligand) and W (affinity between the corresponding wildtype protein and the same ligand) is a better target in a way analogous to transforming Q function to advantage function in reinforcement learning, W here can serve as a natural baseline and subtracting it may reduce the shared variance between M and W, as a result the difference Delta is easier to learn. The authors developed some statistical analysis and did some validation of this idea on some simulated data as well as PLATINUM and DAVIS-Complete dataset.

Critical strengths

The paper is clearly written and the theoretical analysis is sound.

Critical weaknesses

I didn't see a fatal weakness, but the validation could be done more carefully. Both datasets are small, and I see no reason not to combine them, the assay effect could be potentially canceled by M-W and it favors the prediction of the difference. Especially now that the PLATINUM result didn't beat the average difference baseline and no crossover was observed in DAVIS-complete. Also learning is one thing and generalization is another. IMHO for a model to generalize, at least it needs to learn the transferable mutation effect, say you have 3 state ABC, then Delta AB + Delta BC should be close to Delta AC, in this case the anchor can be some reference affinity predictor instead of being tied to the wild type prediction, and currently your anchor is weak so a stronger one could further favor the prediction of the difference.

Questions for authors

See the previous section.

Reviewer_2DRWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper analyzes whether mutant binding affinity should be predicted directly or via the change relative to wild type. It provides simulation of the expected effects and evaluates the approach on PLATINUM and DAVIS-Complete benchmarks.

In short, the paper is composed mostly of fundamental textbook statistics calculations followed by a simple simulation example and a very small set of empirical results of uncertain quality. The paper also does not provide practical, real-life scenario advice for method deployment.

Critical strengths

The idea of investigating advantages of direct vs delta approaches is simple and sensible.

Critical weaknesses

The paper however, suffers from many major issues on several fronts: 1.the best model on PLATINUM seems to be based on the predicting the wild type and adding a constant. This begs the question of whether the models are trained correctly or if the benchmark itself is suitable for the analysis the paper intends to perform. 2. the DAVIS benchmark seems more meaningful, but it does not yield strong result either. There is no statistical significance in benefit at full budget, and there is barely statistical significance on limited budget 3. the training data size can be also significantly confounded since the anchor used by delta route is frozen and uses full data across all data sizes, so it seems like the part of the advantage of the delta model can be attributed to seeing more data, not problem formulation. 4. considering how fundamental the mathematics used to derive those results is, it is concerning how limited the reference list is, it is hard to believe it does justice to the related works. 5. The 3rd takeaway in the introduction seems to be a tautology, not a result: “Modeling ∆ is valuable when mutation-specific affinity changes can be learned beyond the average mutation effect.”

Questions for authors
  1. Provide an explanation on the the wild-type-plus-constant baseline performance on PLATINUM. Is benchmark ill-posed for the type of training being performed or is the model under fitted?
  2. Attempt removing the data size confound from the anchor training (anchors should be trained with different data sizes too to exclude the possibility of data advantage being responsible for the observed gains)
  3. Can the evaluation be expanded to more benchmarks, and can the comparison to state of the art be expanded? Paper in current form does not offer practical guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.