- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Protein-ligand binding affinity prediction upon mutation can be predicted in absolute terms or in relative terms (with respect to a wildtype protein). This papers compares the two formulations by deriving an MSE decomposition, a linear-Gaussian finite-sample analysis, simulations, and an empirical study on two datsets PLATINUM and DAVIS-Complete.
Critical strengths
Clear and well-motivated research question: The paper studies a practically relevant target-design choice: whether mutant affinity should be predicted directly or reconstructed from a predicted mutation effect and a WT-affinity estimate. It uses a principled theoretical framework with a mathematical characterization of the trade-offs between the two routes.
The authors combine controlled simulations with experiments on two real-world datasets, PLATINUM and DAVIS-Complete
Carefully matched comparison: Direct and Delta learners use the same inputs, architecture, training examples, seeds, and nested subset chains, substantially reducing implementation-related differences between the two routes
Critical weaknesses
-
The novelty of the mathematical contribution is unclear. The central derivations use a variance identity for paired differences, squared-error decompositions, and expected OLS risk under a linear-Gaussian model. The paper does not clarify which of these results are intended to be novel or position them relative to the extensive literature on paired designs, change scores versus baseline-adjusted regression, control variates, and multi-fidelity or delta learning. The Related Work section is insufficient and largely restricted to molecular prediction.
-
The practical scope of the absolute-versus-relative comparison is unclear. The paper frames this as absolute vs relative prediction; it is unclear whether this dichotomy is practically meaningful as formulated. Existing methods already combine absolute and relative supervision (MuPIPR Zhou et. al., Twin Peaks Chowdhury et. al., Boltz-2 Passaro et. al. …), and the manuscript’s may benefit from analyzing a continuum of hybrid objectives. More importantly, the experiments do not establish that choosing between the two endpoints has a robust practical consequence: the conclusions rely on a simple, shallow architecture without comparison to established methods, a single split, and evaluations that do not adequately control for protein-family or ligand similarity. Consequently, the results may characterize behavior in this particular experimental setup rather than establish a general principle for mutant binding-affinity prediction.
-
The fixed split does not control protein or ligand similarity. The WT-ligand context split prevents the exact same WT-ligand context from appearing across splits, but it does not enforce separation by protein family, protein-sequence similarity, structural similarity, or ligand similarity. The supplement explicitly allows ligand identities to occur across partitions, and no Tanimoto-similarity threshold is reported. Evaluation examples may contain proteins closely related to training proteins or ligands identical or highly similar to training ligands.
Although the Direct and Delta routes are evaluated on matched examples, this potential leakage does not guarantee that such similarity benefits them equally. The Direct model, Delta model, and frozen WT anchor may exploit protein-family and ligand similarity differently. Therefore, the reported comparison establishes which route performs better under this particular partition, but not whether the difference persists for unseen protein families or ligand chemotypes. These factors may also confound the training-size trends. Increasing the training budget may therefore change dataset composition as well as sample size. Multiple seeds, randomized training subsets, and bootstrap intervals remain conditional on the same partition and do not measure sensitivity to split selection. The main conclusions should therefore be tested using repeated protein-sequence-clustered and ligand-scaffold/Tanimoto-clustered splits. -
The empirical evidence is too narrow. No real-data crossover is observed: PLATINUM shows only a narrowing gap, while DAVIS shows no resolved route trend. The conclusions also depend on one untested architecture, one frozen-anchor construction, and one training protocol. Yet the route advantage is explicitly a function of the Direct learner, Delta learner, anchor, and their interaction. The authors make sure to have a fair compairison across the two routes in terms of architecture. However, comparing the trained model to known baselines or starting from known architectures and adapting them to this analysis would provide a more practical analysis that is not restricted to the very specific architectural decisions made on the paper. The paper does not report a constant mutant-affinity baseline so the absolute competence of the models cannot be assessed. On PLATINUM, WT plus mean outperforms WT plus learned ; thus the route advantage does not demonstrate successful mutation-specific learning.
Questions for authors
- How were the 370 and 831 examples selected, what type of mutations, are they all on the interface with the ligand, etc..?
- Do the conclusions hold under repeated protein-sequence-cluster and ligand-scaffold/Tanimoto-clustered splits?
- Do route differences remain after stratifying or adjusting for protein family, ligand properties, mutation location and class?
- Consider a continuum experiment with some interpolated training target or weighted loss objective.