- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GCMatch for geometry–chemistry dual-similarity pocket matching. It learns fine-grained representations of cavity geometric points and chemical functional points on a neighborhood relation graph, applies separate multi-vector matching, and combines the scores through product fusion. The authors also construct a four-quadrant dataset for evaluation.
Critical strengths
-
The paper is clearly presented, and the method incorporates domain knowledge through a straightforward design.
-
GCMatch achieves the highest performance among the evaluated methods. The component ablations provide useful support for the design choices.
Critical weaknesses
-
Existing methods such as DeeplyTough and BindSiteS-CNN already use geometric and chemical features as input. Clarifying when their single-vector representations obscure local geometric or chemical differences would strengthen the motivation.
-
The paper does not provide information on repeated runs, making it difficult to assess the stability of the performance gains.
-
GCMatch is evaluated only with GVP-GNN. It would be helpful to examine whether the benefits of fine-grained representations and multi-vector matching also hold with another encoder.
Questions for authors
-
What experimental evidence supports the claim that single-vector pocket representations obscure local geometric or chemical differences? Could you provide concrete examples comparing a single-vector baseline with GCMatch?
-
Were the main results and key ablations repeated with different random seeds? Please report the number of independent runs and the mean and standard deviation.
-
Could you replace GVP-GNN with another encoder and evaluate whether GCMatch still outperforms the baseline methods?
7 comments
Response to Question 2
We repeated the main experiments and three key ablations with three independent random seeds. For each method, only the random seed changed across runs. The dataset, splits, and other settings remained fixed. We report the mean ± sample standard deviation (n=3).
1. Main results
As the main results table below shows, GCMatch achieves a mean Full-AUC of 0.921, exceeding the strongest baseline, OneProt (0.873), by 0.048. This gap exceeds their seed standard deviations (0.009 and 0.007). GCMatch also achieves the highest mean in every other column. These results support the stability of its overall performance advantage across three seeds.
Main results. Learning-based methods report the mean ± sample standard deviation over three seeds.
| Method | Full-AUC | Easy | HARD-G | HARD-C | G/C mean |
|---|---|---|---|---|---|
| APoc | 0.822 | 0.935 | 0.911 | 0.619 | 0.765 |
| SiteMotif | 0.822 | 0.915 | 0.898 | 0.652 | 0.775 |
| G-LoSA | 0.817 | 0.915 | 0.898 | 0.639 | 0.769 |
| GCMatch | 0.921 ± 0.009 | 0.955 ± 0.009 | 0.912 ± 0.017 | 0.895 ± 0.003 | 0.904 ± 0.010 |
| OneProt | 0.873 ± 0.007 | 0.908 ± 0.007 | 0.869 ± 0.002 | 0.842 ± 0.012 | 0.856 ± 0.007 |
| ProFSA | 0.859 ± 0.002 | 0.917 ± 0.003 | 0.768 ± 0.003 | 0.892 ± 0.001 | 0.830 ± 0.002 |
| DeeplyTough | 0.831 ± 0.013 | 0.874 ± 0.016 | 0.820 ± 0.007 | 0.799 ± 0.021 | 0.809 ± 0.011 |
| BindSiteS-CNN | 0.775 ± 0.014 | 0.841 ± 0.010 | 0.734 ± 0.014 | 0.751 ± 0.017 | 0.743 ± 0.016 |
Note: APoc, SiteMotif, and G-LoSA require no training. Their results do not vary with the training seed, so we report a single evaluation.
The classification results and external benchmark comparisons are reported in Appendix E, Table 13, and Appendix I, Table 22, respectively. GCMatch achieves the highest mean Accuracy, Precision, Specificity, and MCC. It also achieves the highest mean ROC-AUC in every external benchmark column. These results further support its performance advantage across different metrics and external datasets.
2. Key ablations
As the table below shows, removing cross-type edges reduces Full-AUC from 0.921 to 0.737. Replacing them with the same number of nearest same-type neighbors for each node raises Full-AUC to 0.777. This remains 0.144 below the full model. Node-level matching exceeds type pooling and branch pooling by approximately 0.024 and 0.040, respectively. Using only Y supervision scores approximately 0.039 below full G/C/Y supervision. These gaps exceed the corresponding seed standard deviations. Type pooling exceeds branch pooling by approximately 0.016. This gap is comparable to the branch pooling standard deviation of 0.010, indicating a smaller advantage between the two pooling schemes. These results support the stability of the benefits from cross-type connections, node-level matching, and branch supervision across seeds.
Full-AUC of key ablations over three seeds (n=3).
| Ablation | Setting | Full-AUC |
|---|---|---|
| Graph connectivity | Full graph with same-type and cross-type edges | 0.921 ± 0.009 |
| Graph connectivity | Remove cross-type edges | 0.737 ± 0.020 |
| Graph connectivity | Remove cross-type edges and replace them with an equal number of nearest same-type neighbors per node | 0.777 ± 0.045 |
| Representation granularity | Branch pooling | 0.881 ± 0.010 |
| Representation granularity | Type pooling | 0.897 ± 0.002 |
| Representation granularity | Node-level matching (full model) | 0.921 ± 0.009 |
| Branch supervision | Y supervision only | 0.882 ± 0.010 |
| Branch supervision | G/C/Y supervision (full model) | 0.921 ± 0.009 |
We have marked the repeated-run protocol in Section 4.2 and the updated results in Table 1 in Section 4.3 in blue. The key ablation results in Section 4.4 and Appendix G are also marked in blue. The updated classification metrics appear in blue in Appendix E, Table 13. The external evaluation results are marked in blue in Table 2 in Section 4.3 and in Appendix I, Table 22. We also reduced Figure 2 from six panels to three. Original panels (a), (c), and (f) are now (a), (b), and (c). The other three plots are replaced by three-seed results in the new Table 3. Appendix F retains the single-run results.
During the additional experiments with multiple seeds, we found that the original ProFSA and DeeplyTough training runs had stopped at epoch 20. The original manuscript therefore reported results from incomplete training runs. We have now completed the planned 50 epochs and corrected the results. The results reported here and in the revised manuscript for these two methods are the corrected results from three seeds. We apologize for this oversight.
Response to Question 3
We replaced GVP-GNN with PaiNN [1]. We kept the graph construction, node features, matching procedure, and supervision unchanged, adapting only the encoder inputs. The four direction vectors of each node pass through a linear projection without bias to initialize PaiNN’s vector channels. The 23 scalar features pass through a separate linear projection. Edge features remain 16 Gaussian radial basis functions with centers spanning 0–15 Å. We ran each of the three matching granularities with three independent random seeds. We report the mean ± sample standard deviation (n=3).
1. Comparison with baselines after replacing the encoder
As the table below shows, GCMatch with PaiNN achieves a Full-AUC of 0.916 ± 0.006. This exceeds the strongest baseline, OneProt (0.873 ± 0.007), by approximately 0.043. On HARD-G, it reaches 0.912, above the best learning-based baseline, OneProt (0.869). On HARD-C, it reaches 0.891, comparable to the best learning-based baseline, ProFSA (0.892). These results show that GCMatch retains its overall performance advantage after replacing GVP-GNN with PaiNN.
Comparison with baselines after replacing the encoder (n=3).
| Method | Full-AUC | Easy | HARD-G | HARD-C | G/C mean |
|---|---|---|---|---|---|
| GCMatch (GVP-GNN, node-level) | 0.921 ± 0.009 | 0.955 ± 0.009 | 0.912 ± 0.017 | 0.895 ± 0.003 | 0.904 ± 0.010 |
| GCMatch (PaiNN, node-level) | 0.916 ± 0.006 | 0.947 ± 0.007 | 0.912 ± 0.008 | 0.891 ± 0.008 | 0.901 ± 0.006 |
| OneProt | 0.873 ± 0.007 | 0.908 ± 0.007 | 0.869 ± 0.002 | 0.842 ± 0.012 | 0.856 ± 0.007 |
| ProFSA | 0.859 ± 0.002 | 0.917 ± 0.003 | 0.768 ± 0.003 | 0.892 ± 0.001 | 0.830 ± 0.002 |
| DeeplyTough | 0.831 ± 0.013 | 0.874 ± 0.016 | 0.820 ± 0.007 | 0.799 ± 0.021 | 0.809 ± 0.011 |
| BindSiteS-CNN | 0.775 ± 0.014 | 0.841 ± 0.010 | 0.734 ± 0.014 | 0.751 ± 0.017 | 0.743 ± 0.016 |
2. Comparison of matching granularities with both encoders
As the table below shows, node-level matching also achieves the highest Full-AUC with PaiNN. It exceeds type pooling and branch pooling by approximately 0.018 and 0.022, respectively. The corresponding gaps with GVP-GNN are 0.024 and 0.040. With PaiNN, type pooling slightly exceeds branch pooling by approximately 0.004. These results support the benefits of fine-grained node representations and multi-vector matching with both encoders.
Comparison of matching granularities with both encoders (n=3).
| Encoder | Matching granularity | Full-AUC | Easy | HARD-G | HARD-C | G/C mean |
|---|---|---|---|---|---|---|
| GVP-GNN | Branch pooling | 0.881 ± 0.010 | 0.914 ± 0.007 | 0.873 ± 0.014 | 0.859 ± 0.018 | 0.866 ± 0.012 |
| GVP-GNN | Type pooling | 0.897 ± 0.002 | 0.929 ± 0.002 | 0.894 ± 0.001 | 0.869 ± 0.004 | 0.882 ± 0.003 |
| GVP-GNN | Node-level matching | 0.921 ± 0.009 | 0.955 ± 0.009 | 0.912 ± 0.017 | 0.895 ± 0.003 | 0.904 ± 0.010 |
| PaiNN | Branch pooling | 0.894 ± 0.004 | 0.920 ± 0.006 | 0.890 ± 0.009 | 0.873 ± 0.012 | 0.881 ± 0.005 |
| PaiNN | Type pooling | 0.898 ± 0.008 | 0.928 ± 0.010 | 0.896 ± 0.008 | 0.870 ± 0.008 | 0.883 ± 0.007 |
| PaiNN | Node-level matching | 0.916 ± 0.006 | 0.947 ± 0.007 | 0.912 ± 0.008 | 0.891 ± 0.008 | 0.901 ± 0.006 |
We have added the main findings from replacing the encoder in blue in Section 4.4 of the revised manuscript. The input adaptation for PaiNN and the complete results in Appendix J (Table 23) are also marked in blue.
References
[1] Schütt KT, Unke OT, Gastegger M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. ICML, 2021: 9377–9388.
Response to Question 1
Thank you for your careful review and helpful comments. We have added experiments to address your questions.
We provide evidence from a concrete example, prior work, and overall performance comparisons.
1. A concrete example
We use the test-set pocket pair 4CYJ–4FL2 to compare how single-vector models and GCMatch respond to local changes. We fix 4FL2 and modify 4CYJ in three ways:
We compare GCMatch, its single-vector variant, DeeplyTough, and BindSiteS-CNN. Each model's similarity scores are converted to percentiles of its validation POS scores. The dashed line marks the threshold at 90% POS recall, corresponding to the 10th percentile.
As Figure 3 in Appendix K of the revised manuscript shows:
On this pocket pair, the single-vector models respond less to local side-chain mutations and displacements. Local differences may therefore be more easily obscured. GCMatch's node-level matching reflects these changes in its scores. The coordinate noise control does not produce the same threshold-crossing behavior.
2. Related findings in prior work
Similar findings have been reported in protein homolog retrieval. Cohn et al. [1] note that pooling residue embeddings into a single vector may obscure local motifs and conserved residues. With the same backbone and contrastive objective, mean pooling gives lower cR@1 than residue-level MaxSim. The gaps are about 0.26 on SCOPe and 0.15 on Pfam.
3. Comparison with the single-vector baselines named in your review
We also compare GCMatch with BindSiteS-CNN and DeeplyTough. The results below are from Table 1 of the revised manuscript. They report the mean and sample standard deviation over three seeds (n=3). Full is the AUC over all test pairs. G and C are the AUCs for POS versus HARD-G and HARD-C, respectively.
GCMatch outperforms both single-vector baselines on Full-AUC, HARD-G, and HARD-C in this task.
Scope of the evidence and manuscript changes
The alanine substitutions in the first experiment change both chemistry and local shape. We cannot precisely separate their effects. The third comparison supports GCMatch's performance advantage in this task, but does not directly establish that single-vector representations obscure local differences. The concrete example shows that single-vector models retain relatively high similarity scores under the local changes tested. We therefore limit our conclusion to the claim that single-vector representations may obscure local pocket differences.
We have added the case study in Appendix K, with its description and Figure 3 caption marked in blue. The narrowed claims in the Abstract and Section 1 are also marked in blue. The updated results over three seeds appear in blue in Table 1 in Section 4.3.
Reference
[1] Cohn G, Gumaste R, Hoang M, Lakshman V. PROTOCOL: Late Interaction Retrieval for Protein Homolog Search. arXiv:2605.29158, 2026.