- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors propose a faster way to calculate protein surface meshes with AlphaSurf, discarding the slow MSMS triangulated molecular surface generator.
Critical strengths
AlphaSurf is a protein surface mesh generation method that is substantially faster than MSMS and EDTSurf. Fig 4 shows that AlphaSurf is orders of magnitude faster than MSMS, EDTSurf, and NanoShaper. In addition, AlphaSurf is much more usable at 1% compared to the aforementioned methods. It appears that AlphaSurf retains the performance of other methods while being much faster. The priors of AlphaSurf are strong - one does not need to render the dense surface mesh if not using the mesh for visualization, thus yielding a speedup if only using it for feature propagation.
Critical weaknesses
"I am not fully convinced by the analyses in Fig 6. Aside from MaSIF-Ligand and AF2 hetero, it appears that AlphaSurf is within error or worse compared to methods that fall under the non shaded areas. Do error bars denote SEM, SD, or confidence intervals?
Looking at supplemental tables S1 and S2, MSMS+simplification is reasonably fast. The main text highlights 10x improvement of AlphaSurf on PINDER, but the supplemental table shows a much smaller speedup. EDTSurf at gs=0.4 is also quite fast and has comparable PINDER processing times to AlphaSurf, can the authors elaborate on what gs a layperson should look at to determine speedup?
The paper uses a lot of technical jargon, making reading difficult, especially if one is not well-versed in the field of surface mesh calculation. I am not an expert in this area, and understanding the latter parts of the introduction and the method itself is quite difficult."
Questions for authors
Suggestion: the x axis labels in Figure 4 are difficult to read and are messy, perhaps cleaning up the formatting and making it clear which colors correspond to MSMS, EDTSurf, NanoShaper would be helpful in understanding this figure.
3 comments
Answer to reviewer 3 (2/2)
W2.
W2. Looking at supplemental tables S1 and S2, MSMS+simplification is reasonably fast. The main text highlights 10x improvement of AlphaSurf on PINDER, but the supplemental table shows a much smaller speedup. EDTSurf at gs=0.4 is also quite fast and has comparable PINDER processing times to AlphaSurf, can the authors elaborate on what gs a layperson should look at to determine speedup?
A-W2. Apologies for this confusion: as explained in our general comments and in the revised manuscript, one needs to consider separately the preprocessing time and the learning time. Our 10x improvement is for the former, but it still translates into a 2-3× speedup of the whole learning pipeline versus MSMS. EDTSurf's cost depends on its grid scale: at gs = 0.4, where its meshes match ours in size, it is comparably fast, but this grid scale must be tuned dataset-wise in order to be on the fly, whereas ours doesn't necessitate any tuning. At this matched cost, AlphaSurf is more robust (0% vs close to 1% failures) and more accurate on Apo-Hetero, AF2-Hetero and MISATO (see W1).
W3.
W3. The paper uses a lot of technical jargon, making reading difficult, especially if one is not well-versed in the field of surface mesh calculation. I am not an expert in this area, and understanding the latter parts of the introduction and the method itself is quite difficult.
A-W3. We acknowledge that the alpha-complex is a non-trivial construction from computational geometry, which does not belong to the usual background of ML practitioners, and we rely on precise mathematical terms to describe molecular surfaces and the associated objects unambiguously. We did our best to provide a gentle and illustrated introduction to it in the main text, and the revised version reorganizes the first section to start with learning rather than surface representations, which should make it easier to follow.
Q1.
Q1. Suggestion: the x axis labels in Figure 4 are difficult to read and are messy, perhaps cleaning up the formatting and making it clear which colors correspond to MSMS, EDTSurf, NanoShaper would be helpful in understanding this figure.
A-Q1. We acknowledge that this figure wasn't as clear as it could be; we provide a revised version (for MaSIF-Ligand also) which makes the numbers and the legend more understandable.
We have tried to address all of the comments and questions by the reviewer, and thank the reviewer again for helping us improve our submission. We invite the reviewer to also consider the general feedback we provided in our general comment to all reviewers. We will make sure to address all comments or concerns in the potential final version of our paper.
Follow up to reviewer 3
We thank you again for your review and your remarks, which we believe have made the paper better.
As the discussion period closes soon, we would be grateful to know whether our answers above address your questions. We believe they do, and we would be happy to clarify any remaining points.
Answer to reviewer 3 (1/2)
Thank you for your detailed comments. We would like to draw your attention to the general answer, in which we resume the additional results obtained for the rebuttal. Concerning your points, we corrected our PINDER runs, which now show clear gains over EDTSurf on the Apo-Hetero and AF2-Hetero splits, added a comparison with EDTSurf on molecular dynamics (MISATO), revised Figure 4, and reorganized the first section to make the paper easier to follow. In addition, we provide a point-by-point answer to your comments below.
W1.
A-W1. First, the error bars denote standard deviation over 3 seeds.
The non-shaded area corresponds to EDTSurf, used with much smaller grid sizes than its default. We agree that it represents a reasonable baseline but would like to highlight that our work is the first to assess its performance for learning applications. The revised version adds three results where AlphaSurf is clearly better than EDTSurf at grid scale 0.4:
PINDER-Pair AUROC (%), mean ± std. Full results in Figure 6.
Holo-conformation training
Random-frame training
Binding-site prediction on MISATO (%, mean ± std), evaluated on holo conformations. Bold: best surface in each training setting.