- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
PEAR predicts protein–ligand binding affinity using a shared encoder for bound and nominally unbound graphs, followed by paired readouts at atom, edge, and pocket levels. Evaluations on CleanSplit, ATOM3D LBA, and held-out pocket clusters show competitive results without pretraining. However, the unbound-branch definition and attribution of the improvements need clarification.
Critical strengths
- The paper addresses a relevant question: how to combine corresponding structural representations before graph-level pooling.
- The authors provide comparisons of pairing operations, component ablations, repeated runs, and evaluations across several protocols.
- Some configurations are competitive without pretraining, and the paper appropriately notes that the LBA30 improvement over CheapNet is not decisive.
Critical weaknesses
- The unbound branch is unclear. Appendix D (p. 16, Eqs. 18–19) describes edge-state updates on intermolecular node pairs and feedback in both branches. This appears to allow protein–ligand information exchange despite removing intermolecular message-passing edges. The authors should clarify the implementation and its interpretation.
- The ablations do not fully separate paired readouts from added capacity or richer local features. A bound-only model with the same readout structure and parameter-matched local MLP controls would help. The preliminary graph-level concatenation test also uses a different backbone from PEAR.
- Generalization gains vary by setting. PEAR slightly improves CASF-2016 RMSE over GEMS (1.258 vs. 1.274), but both PEAR variants have worse RMSE on the independent subset, and pre-finetuning results on held-out clusters trail GEMS. All three evaluation protocols use PDBbind-derived data.
- Uncertainty and model selection need clearer reporting. The paper does not provide paired confidence intervals for small differences, and pocket-cluster finetuning uses one fixed 25-example support set per cluster. The selection process for model variants should also be clarified.
- The existing figures classify prior work but do not show PEAR’s pipeline. Please add a method diagram showing both branches, edge feedback, paired readouts, and the prediction head, consistent with the equations and algorithm.
Questions for authors
- Does the unbound branch apply intermolecular edge feedback? What changes when all cross-molecule communication is disabled?
- Can you provide parameter-matched and bound-only controls, and clarify whether architecture choices were made using validation data only?
3 comments
3. On the preliminary experiment. The purpose of our preliminary experiment (Table 1) was to show that the vanilla MPNN does not benefit from the extra unbound copy in the CleanSplit dataset. We believe the experiment fulfills this purpose and provides a basis for our exposition on paired readouts. The backbones within this experiment are matched to each other in terms of architecture and number of parameters (only the head MLP input widths differ by necessity*) to isolate the difference between a one-copy network and a two-copy network over a small range of and layer values, so they are consistent internally. The difference between the vanilla neural networks and the backbone of PEAR is that PEAR has an additional edge feedback mechanism on persistent edge states during message passing. We include this in our ablation because "decontaminated" datasets such as CleanSplit are relatively new territory, so we believe identifying basic components of graph neural networks that improve performance is beneficial to the field. However, we acknowledge that this produces a logical fork in the sense that the benefits of paired readouts may appear to be overstated. We have since confirmed our exploratory finding by running the backbone with edge states and edge feedback but no paired readouts.
| Backbone | Paired readouts | Params | CASF-2016 RMSE | paired vs PEAR | 95% CI |
|---|---|---|---|---|---|
| vanilla | no | 1,345,233 | 1.323 | +0.065 | [+0.025, +0.109] |
| vanilla + edge states and feedback | no | 1,497,745 | 1.371 | +0.113 | [+0.076, +0.151] |
| vanilla + edge states and feedback (PEAR) | yes | 1,682,897 | 1.258 | — | — |
The first two rows share the absence of paired readouts, so their difference would isolate the edge mechanism, which costs 0.048 RMSE, 95% CI [+0.014, +0.081]. The second and third rows share a backbone, so their difference isolates the paired readouts, which are worth 0.113 RMSE, 95% CI [+0.076, +0.151]. These values are derived five fold CV ensembles, which exhibit a gain in accuracy from ensembling and so are in general lower than numbers in the ablation table.
PEAR outperforms the no-paired-readout variation over values 0, 1 and 2.
* the two-copy prediction head takes twice the number of input channels and hence has ~3.5-8% more parameters than the one-copy network depending on the number of layers.
Please continue to the next comment for the third part of our response.
4. On added capacity. It is true that a candidate reason for improved performance on these tasks is simply more capacity through added parameters in the paired readouts. Several networks in our submitted draft already do not perform better than models with fewer parameters. The two-copy networks in Table 1 have 3.5-8% more parameters than their one-copy counterparts. The "no-paired-readout" models in Table 3 gain 4.5% and 6.7% parameters as goes from 0 to 1 and 2, with RMSE of 1.386, 1.391 and 1.386.
- Because paired readouts add 25% more parameters, we obtain five fold CV RMSE and R values for a backbone model that has increased width to match the number of parameters of PEAR (1.68M). The model actually performed no better than the ensemble original backbone, at an RMSE of 1.347 vs. an RMSE of 1.323. Together with a PEAR model with intermolecular edges removed from the bound copy (RMSE of 1.389), these models are the three worst five-fold ensembles we've trained. Please see the following table:
| Model | Paired readouts | Params | CASF-2016 RMSE | R |
|---|---|---|---|---|
| PEAR | yes | 1,682,897 | 1.258 | 0.834 |
| Backbone, width-matched to PEAR | no | 1,682,787 | 1.347 | 0.803 |
| PEAR, intermolecular edges removed from bound copy | yes | 1,682,897 | 1.389 | 0.776 |
| Backbone, original width | no | 1,345,233 | 1.323 | 0.814 |
All rows are five fold CV ensembles on CASF-2016.
5. On bound-only and MLP control. We have run a parameter-matched experiment where the atom paired readout is replaced by a per-node MLP on only the bound copy of the complex. It reaches a 5-fold CV RMSE of 1.276 against 1.258 for PEAR as reported. The Pearson R is 0.830 against 0.834 in PEAR. Both paired 95% CIs include zero; see below.
- As one can see, a per-node MLP on the bound copy can perform similarly well on cleansplit. For our interpretation of this result, see item 2.1.3 of our global comment, "On the fit of our work within the binding affinity prediction field".
| Params | CASF-2016 RMSE | Pearson R | |
|---|---|---|---|
| PEAR | 1,682,897 | 1.258 | 0.834 |
| Bound-only per-node MLP | 1,688,618 | 1.276 | 0.830 |
| paired difference | +0.3% | +0.018 | −0.004 |
| 95% CI | — | [−0.005, +0.043] | [−0.014, +0.006] |
Paired bootstrap over the 285 CASF-2016 complexes, 10,000 resamples.
6. On the fit of our work within the binding affinity prediction field. Please see our global comment.
We thank the reviewer for the detailed comments. We ran several new experiments in response and report them below.
1. On removing intermolecular edges. Adhering to this reporting scheme, we have the following result when intermolecular edges are removed from the bound copy in PEAR, equivalent to running the model on two "unbound" masked copies of the molecule. We find that the lower and higher ends of the 95% paired confidence interval are worse by 0.063 and 0.200 RMSE, respectively:
CASF-2016, 5-fold CV ensemble, 285 complexes, 10,000 bootstrap resamples.
2. Clarifying how persistent edge states and edge feedback affect the unbound copy of the complex. There is no message passing between the protein and the ligand in the unbound copy. After the preliminary experiment, we added an edge feedback module that does scatter an edge representation back to the pocket and the ligand, in a more restricted way than equivariant message passing does, namely it is symmetric, scalar-only, and without geometry (i.e. spherical harmonics), as described in the appendix. During exploratory experiments, we found that this mechanism does not improve the backbone itself. Because it wasn't included in the final set of modules + hyper parameters in the vanilla network, we did not report its five-fold CV or five-seed averages as an independent backbone model. It did, however, later improve the performance of paired readouts that operated on edges. We therefore used both persistent edges and edge feedback in PEAR and performed an ablation of it accordingly. The bound and unbound copies still produce distinct representations due to message passing at the interface in the bound copy. However, we recognize that this can on the surface be perceived as contrary to the physical heuristic that we allude to throughout the paper, i.e. a strictly unbound copy of the complex should have zero communication across the interface. However, the basis for using the unbound copy comes from its common usage as a data processing step in the literature and their empirical benefits, and we believe any improvements to the backbone that still maintains different representations between copies should be utilized, even if they risk infringing on the bound/unbound distinction on a heuristic level. Indeed, unlike physical symmetries, we have no mathematical need to enforce a specific definition of the unbound copy. For example, using real apo conformations is outside the scope of this paper (and more involved considering datasets such as apobind do not have full PDBBind coverage), but if we were to use them, we can still apply our edge feedback mechanism. In the final draft, we will make it clear that 1. the edge feedback mechanism does exchange some information across the interface in a more restricted way than equivariant message passing and 2. the ablation suggests the edge paired readout does not perform as well if there is no edge feedback.
Please continue to the next comment for the second part of our response.