acceptodds
Under review as a conference paper at ICLR 2027

GCMatch: Protein Pocket Matching with Geometric–Chemical Spatial Semantic Modeling and Fine-Grained Representations

Abstract

Protein pocket similarity matching supports drug repurposing and pocket function annotation. This task requires similarity in both cavity geometry and chemical environment, i.e., dual similarity. Existing learning-based methods typically encode geometric and chemical information jointly within the same local unit. They produce single-vector pocket representations for matching, enabling effective assessment of overall similarity. However, they do not explicitly model spatial semantic relations between cavity geometry and chemical environment, and the coarse-grained single-vector representation obscures local geometric or chemical differences. Moreover, the datasets used by these methods define similarity labels according to whether pockets bind similar ligands, but ligand similarity is not equivalent to dual similarity. Together, these issues limit the accuracy of dual-similarity pocket matching. We propose GCMatch, a dual-similarity pocket matching method based on spatial semantic modeling and fine-grained representations. First, GCMatch extracts cavity geometric points and chemical functional points from each pocket. It then connects both point types in a single neighborhood relation graph. A graph encoder learns two sets of fine-grained node representations, one for geometry and one for chemistry. Multi-vector matching is applied separately to each set, and product fusion produces the final matching score. GCMatch achieves a test AUC of 0.920, exceeding the best baseline by 5.1 percentage points. We also construct a four-quadrant dataset with separate geometric and chemical similarity labels for each pocket pair. It serves as a unified evaluation benchmark for the field.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Pocket matching, Graph neural networks, Representation learning.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HMMGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GCMatch for geometry–chemistry dual-similarity pocket matching. It learns fine-grained representations of cavity geometric points and chemical functional points on a neighborhood relation graph, applies separate multi-vector matching, and combines the scores through product fusion. The authors also construct a four-quadrant dataset for evaluation.

Critical strengths
  1. The paper is clearly presented, and the method incorporates domain knowledge through a straightforward design.

  2. GCMatch achieves the highest performance among the evaluated methods. The component ablations provide useful support for the design choices.

Critical weaknesses
  1. Existing methods such as DeeplyTough and BindSiteS-CNN already use geometric and chemical features as input. Clarifying when their single-vector representations obscure local geometric or chemical differences would strengthen the motivation.

  2. The paper does not provide information on repeated runs, making it difficult to assess the stability of the performance gains.

  3. GCMatch is evaluated only with GVP-GNN. It would be helpful to examine whether the benefits of fine-grained representations and multi-vector matching also hold with another encoder.

Questions for authors
  1. What experimental evidence supports the claim that single-vector pocket representations obscure local geometric or chemical differences? Could you provide concrete examples comparing a single-vector baseline with GCMatch?

  2. Were the main results and key ablations repeated with different random seeds? Please report the number of independent runs and the mean and standard deviation.

  3. Could you replace GVP-GNN with another encoder and evaluate whether GCMatch still outperforms the baseline methods?

7 comments
AuthorsOpenReview ↗

Response to Question 1

Thank you for your careful review and helpful comments. We have added experiments to address your questions.

We provide evidence from a concrete example, prior work, and overall performance comparisons.

1. A concrete example

We use the test-set pocket pair 4CYJ–4FL2 to compare how single-vector models and GCMatch respond to local changes. We fix 4FL2 and modify 4CYJ in three ways:

  • (a) Cumulative alanine substitutions: Contact residues are mutated to alanine one by one.
  • (b) Coordinate noise control: Independent coordinate noise of 0.2 Å is added.
  • (c) Random side-chain displacement: Residue types are preserved. The side chains of 10 residues are each displaced along a random direction. The displacement increases from 0 to 5.0 Å, with no systematic change in pocket size.

We compare GCMatch, its single-vector variant, DeeplyTough, and BindSiteS-CNN. Each model's similarity scores are converted to percentiles of its validation POS scores. The dashed line marks the threshold at 90% POS recall, corresponding to the 10th percentile.

As Figure 3 in Appendix K of the revised manuscript shows:

  • In (a), GCMatch first falls below the threshold at the fourth substitution. DeeplyTough first falls below it at the tenth. The single-vector variant and BindSiteS-CNN remain at the 43rd and 47th percentiles after the tenth substitution.
  • In (b), no model falls below the threshold.
  • In (c), GCMatch falls below the threshold at 2.5 Å and remains below it. At 5.0 Å, the single-vector variant, DeeplyTough, and BindSiteS-CNN remain at the 24th, 42nd, and 33rd percentiles.

On this pocket pair, the single-vector models respond less to local side-chain mutations and displacements. Local differences may therefore be more easily obscured. GCMatch's node-level matching reflects these changes in its scores. The coordinate noise control does not produce the same threshold-crossing behavior.

2. Related findings in prior work

Similar findings have been reported in protein homolog retrieval. Cohn et al. [1] note that pooling residue embeddings into a single vector may obscure local motifs and conserved residues. With the same backbone and contrastive objective, mean pooling gives lower cR@1 than residue-level MaxSim. The gaps are about 0.26 on SCOPe and 0.15 on Pfam.

3. Comparison with the single-vector baselines named in your review

We also compare GCMatch with BindSiteS-CNN and DeeplyTough. The results below are from Table 1 of the revised manuscript. They report the mean and sample standard deviation over three seeds (n=3). Full is the AUC over all test pairs. G and C are the AUCs for POS versus HARD-G and HARD-C, respectively.

MethodFullGC
BindSiteS-CNN (single vector)0.775 ± 0.0140.734 ± 0.0140.751 ± 0.017
DeeplyTough (single vector)0.831 ± 0.0130.820 ± 0.0070.799 ± 0.021
GCMatch (multiple vectors, full model)0.921 ± 0.0090.912 ± 0.0170.895 ± 0.003

GCMatch outperforms both single-vector baselines on Full-AUC, HARD-G, and HARD-C in this task.

Scope of the evidence and manuscript changes

The alanine substitutions in the first experiment change both chemistry and local shape. We cannot precisely separate their effects. The third comparison supports GCMatch's performance advantage in this task, but does not directly establish that single-vector representations obscure local differences. The concrete example shows that single-vector models retain relatively high similarity scores under the local changes tested. We therefore limit our conclusion to the claim that single-vector representations may obscure local pocket differences.

We have added the case study in Appendix K, with its description and Figure 3 caption marked in blue. The narrowed claims in the Abstract and Section 1 are also marked in blue. The updated results over three seeds appear in blue in Table 1 in Section 4.3.

Reference

[1] Cohn G, Gumaste R, Hoang M, Lakshman V. PROTOCOL: Late Interaction Retrieval for Protein Homolog Search. arXiv:2605.29158, 2026.

AuthorsOpenReview ↗

Response to Question 2

We repeated the main experiments and three key ablations with three independent random seeds. For each method, only the random seed changed across runs. The dataset, splits, and other settings remained fixed. We report the mean ± sample standard deviation (n=3).

1. Main results

As the main results table below shows, GCMatch achieves a mean Full-AUC of 0.921, exceeding the strongest baseline, OneProt (0.873), by 0.048. This gap exceeds their seed standard deviations (0.009 and 0.007). GCMatch also achieves the highest mean in every other column. These results support the stability of its overall performance advantage across three seeds.

Main results. Learning-based methods report the mean ± sample standard deviation over three seeds.

MethodFull-AUCEasyHARD-GHARD-CG/C mean
APoc0.8220.9350.9110.6190.765
SiteMotif0.8220.9150.8980.6520.775
G-LoSA0.8170.9150.8980.6390.769
GCMatch0.921 ± 0.0090.955 ± 0.0090.912 ± 0.0170.895 ± 0.0030.904 ± 0.010
OneProt0.873 ± 0.0070.908 ± 0.0070.869 ± 0.0020.842 ± 0.0120.856 ± 0.007
ProFSA0.859 ± 0.0020.917 ± 0.0030.768 ± 0.0030.892 ± 0.0010.830 ± 0.002
DeeplyTough0.831 ± 0.0130.874 ± 0.0160.820 ± 0.0070.799 ± 0.0210.809 ± 0.011
BindSiteS-CNN0.775 ± 0.0140.841 ± 0.0100.734 ± 0.0140.751 ± 0.0170.743 ± 0.016

Note: APoc, SiteMotif, and G-LoSA require no training. Their results do not vary with the training seed, so we report a single evaluation.

The classification results and external benchmark comparisons are reported in Appendix E, Table 13, and Appendix I, Table 22, respectively. GCMatch achieves the highest mean Accuracy, Precision, Specificity, and MCC. It also achieves the highest mean ROC-AUC in every external benchmark column. These results further support its performance advantage across different metrics and external datasets.

2. Key ablations

As the table below shows, removing cross-type edges reduces Full-AUC from 0.921 to 0.737. Replacing them with the same number of nearest same-type neighbors for each node raises Full-AUC to 0.777. This remains 0.144 below the full model. Node-level matching exceeds type pooling and branch pooling by approximately 0.024 and 0.040, respectively. Using only Y supervision scores approximately 0.039 below full G/C/Y supervision. These gaps exceed the corresponding seed standard deviations. Type pooling exceeds branch pooling by approximately 0.016. This gap is comparable to the branch pooling standard deviation of 0.010, indicating a smaller advantage between the two pooling schemes. These results support the stability of the benefits from cross-type connections, node-level matching, and branch supervision across seeds.

Full-AUC of key ablations over three seeds (n=3).

AblationSettingFull-AUC
Graph connectivityFull graph with same-type and cross-type edges0.921 ± 0.009
Graph connectivityRemove cross-type edges0.737 ± 0.020
Graph connectivityRemove cross-type edges and replace them with an equal number of nearest same-type neighbors per node0.777 ± 0.045
Representation granularityBranch pooling0.881 ± 0.010
Representation granularityType pooling0.897 ± 0.002
Representation granularityNode-level matching (full model)0.921 ± 0.009
Branch supervisionY supervision only0.882 ± 0.010
Branch supervisionG/C/Y supervision (full model)0.921 ± 0.009

We have marked the repeated-run protocol in Section 4.2 and the updated results in Table 1 in Section 4.3 in blue. The key ablation results in Section 4.4 and Appendix G are also marked in blue. The updated classification metrics appear in blue in Appendix E, Table 13. The external evaluation results are marked in blue in Table 2 in Section 4.3 and in Appendix I, Table 22. We also reduced Figure 2 from six panels to three. Original panels (a), (c), and (f) are now (a), (b), and (c). The other three plots are replaced by three-seed results in the new Table 3. Appendix F retains the single-run results.

During the additional experiments with multiple seeds, we found that the original ProFSA and DeeplyTough training runs had stopped at epoch 20. The original manuscript therefore reported results from incomplete training runs. We have now completed the planned 50 epochs and corrected the results. The results reported here and in the revised manuscript for these two methods are the corrected results from three seeds. We apologize for this oversight.

AuthorsOpenReview ↗

Response to Question 3

We replaced GVP-GNN with PaiNN [1]. We kept the graph construction, node features, matching procedure, and supervision unchanged, adapting only the encoder inputs. The four direction vectors of each node pass through a linear projection without bias to initialize PaiNN’s vector channels. The 23 scalar features pass through a separate linear projection. Edge features remain 16 Gaussian radial basis functions with centers spanning 0–15 Å. We ran each of the three matching granularities with three independent random seeds. We report the mean ± sample standard deviation (n=3).

1. Comparison with baselines after replacing the encoder

As the table below shows, GCMatch with PaiNN achieves a Full-AUC of 0.916 ± 0.006. This exceeds the strongest baseline, OneProt (0.873 ± 0.007), by approximately 0.043. On HARD-G, it reaches 0.912, above the best learning-based baseline, OneProt (0.869). On HARD-C, it reaches 0.891, comparable to the best learning-based baseline, ProFSA (0.892). These results show that GCMatch retains its overall performance advantage after replacing GVP-GNN with PaiNN.

Comparison with baselines after replacing the encoder (n=3).

MethodFull-AUCEasyHARD-GHARD-CG/C mean
GCMatch (GVP-GNN, node-level)0.921 ± 0.0090.955 ± 0.0090.912 ± 0.0170.895 ± 0.0030.904 ± 0.010
GCMatch (PaiNN, node-level)0.916 ± 0.0060.947 ± 0.0070.912 ± 0.0080.891 ± 0.0080.901 ± 0.006
OneProt0.873 ± 0.0070.908 ± 0.0070.869 ± 0.0020.842 ± 0.0120.856 ± 0.007
ProFSA0.859 ± 0.0020.917 ± 0.0030.768 ± 0.0030.892 ± 0.0010.830 ± 0.002
DeeplyTough0.831 ± 0.0130.874 ± 0.0160.820 ± 0.0070.799 ± 0.0210.809 ± 0.011
BindSiteS-CNN0.775 ± 0.0140.841 ± 0.0100.734 ± 0.0140.751 ± 0.0170.743 ± 0.016

2. Comparison of matching granularities with both encoders

As the table below shows, node-level matching also achieves the highest Full-AUC with PaiNN. It exceeds type pooling and branch pooling by approximately 0.018 and 0.022, respectively. The corresponding gaps with GVP-GNN are 0.024 and 0.040. With PaiNN, type pooling slightly exceeds branch pooling by approximately 0.004. These results support the benefits of fine-grained node representations and multi-vector matching with both encoders.

Comparison of matching granularities with both encoders (n=3).

EncoderMatching granularityFull-AUCEasyHARD-GHARD-CG/C mean
GVP-GNNBranch pooling0.881 ± 0.0100.914 ± 0.0070.873 ± 0.0140.859 ± 0.0180.866 ± 0.012
GVP-GNNType pooling0.897 ± 0.0020.929 ± 0.0020.894 ± 0.0010.869 ± 0.0040.882 ± 0.003
GVP-GNNNode-level matching0.921 ± 0.0090.955 ± 0.0090.912 ± 0.0170.895 ± 0.0030.904 ± 0.010
PaiNNBranch pooling0.894 ± 0.0040.920 ± 0.0060.890 ± 0.0090.873 ± 0.0120.881 ± 0.005
PaiNNType pooling0.898 ± 0.0080.928 ± 0.0100.896 ± 0.0080.870 ± 0.0080.883 ± 0.007
PaiNNNode-level matching0.916 ± 0.0060.947 ± 0.0070.912 ± 0.0080.891 ± 0.0080.901 ± 0.006

We have added the main findings from replacing the encoder in blue in Section 4.4 of the revised manuscript. The input adaptation for PaiNN and the complete results in Appendix J (Table 23) are also marked in blue.

References

[1] Schütt KT, Unke OT, Gastegger M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. ICML, 2021: 9377–9388.

Reviewer_JV6ZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GCMatch, a method for matching protein pocket similarities by integrating geometric and chemical-space semantic information. The authors represent the geometric structure and chemical functional groups of a pocket as two distinct sets of spatial points, employing a graph neural network to learn the spatial relationships between them. Geometric and chemical similarities are then calculated separately using a fine-grained, multi-vector matching approach. Furthermore, the authors constructed a four-quadrant dataset comprising 75,200 pocket pairs to evaluate these two types of similarity—both individually and in combination. On this dataset, GCMatch achieves a Full-AUC of 0.920, representing an improvement over the strongest baseline, OneProt (0.869). Ablation studies further validate the effectiveness of the model's various components.

Critical strengths
  1. The paper addresses the important problem of jointly considering geometric and chemical similarity in protein pocket matching. The proposed Geometry–Chemistry Spatial Modeling and Fine-grained Multi-Vector Matching provide a reasonable approach to capturing spatial relationships while preserving local information. GCMatch achieves a Full-AUC of 0.920, outperforming the strongest baseline, OneProt (0.869). It also performs well on both geometric and chemical hard negatives, achieving AUCs of 0.914 and 0.895 compared to OneProt's 0.878 and 0.830.

  2. I appreciate the four-quadrant dataset design, which separately considers geometric and chemical similarity and introduces two types of challenging negative samples. The authors also carefully control sample distributions to enable more meaningful comparisons. The ablation studies are convincing: removing cross-type edges reduces AUC from 0.920 to 0.721, while node-level representations and additional branch supervision further improve performance. Together with comparisons against multiple baselines and efficiency analysis, these experiments provide solid support for the proposed design.

Critical weaknesses
  1. This is my primary concern regarding the paper. A key premise of the study is that ligand similarity does not necessarily serve as an accurate proxy for the geometric and chemical environment similarity of protein pockets. Nevertheless, the four-quadrant dataset constructed by the authors relies on the ECFP4 Tanimoto similarity between bound ligands as a proxy label for "pocket chemical similarity." Although the authors define geometric and chemical similarity separately and perform validations using alternative molecular fingerprints, these additional fingerprints still measure the chemical similarity of the ligands rather than the chemical environment of the pockets themselves. Consequently, I believe the current validation is insufficient to demonstrate that these labels accurately reflect the true chemical similarity of the pockets. Given that both the training and evaluation of the model depend on these labels, this could complicate the interpretation of the reported performance improvements.
Questions for authors
  1. Regarding the validity of the "chemical similarity ground truth." The paper notes that ligand similarity does not necessarily reflect the chemical similarity of the binding pocket; yet, the dataset still uses the ECFP4 Tanimoto similarity of bound ligands to define the chemical similarity labels. Could the authors provide further justification for this choice? Additionally, could they offer more direct validation—such as by leveraging the physicochemical properties of the pockets themselves, employing methods to compare chemical environments, or utilizing independently annotated pocket similarity data—to confirm the validity of these labels?

  2. Regarding generalizability to other pocket-matching tasks. GCMatch performs strongly on the proposed four-quadrant dataset. Have the authors considered further validation using independent pocket-matching benchmarks or practical pocket-retrieval tasks? Such experiments would help demonstrate whether the geometric and chemical representations learned by the model can generalize to different evaluation tasks and real-world application scenarios.

2 comments
AuthorsOpenReview ↗

Response to Question 1

Thank you for taking the time to review our paper and share your suggestions.

We use the ECFP4 Tanimoto similarity between bound ligands only as a proxy for the chemical aspect of pocket similarity. We label geometric similarity independently using PPS-align. The final matching label requires both geometric and chemical similarity criteria to be met. We chose this chemical proxy mainly because a general tool for directly labeling the similarity of pocket chemical environments is currently lacking. Even comparisons based on protein–ligand interactions depend on both the pocket and the bound ligand [1]. We therefore followed the approach used in TOUGH-M1 [2], which constructs labels from the chemical similarity of bound ligands.

To examine whether this proxy reflects the chemical environments of the pockets themselves, we added two validation experiments.

Both experiments compare geometrically similar pairs with positive chemical labels (POS) against geometrically similar pairs with negative chemical labels (HARD-C). Both groups satisfy the same geometric similarity criterion (G=1). Their geometric similarity distributions were also matched during sampling to reduce the influence of geometric differences.

1. Validation using binding contact environments

We used SSC [2] to analyze the contact environments in observed protein–ligand complexes. As the table below shows, the median SSC is higher for POS than for HARD-C in all three splits. The corresponding ROC-AUC values are 0.952, 0.936, and 0.941. These results show that the two groups defined by the chemical labels also differ in their observed binding contact environments.

SSC comparison between POS and HARD-C.

SplitPOS / HARD-C pairsMedian SSC for POSMedian SSC for HARD-CSSC ROC-AUC
Training15,000 / 15,0000.5430.1960.952
Validation1,800 / 1,8000.5500.2050.936
Test2,000 / 2,0000.5640.1910.941

2. Independent validation using pocket chemical environments

We further used PocketFEATURE [3] for independent validation based on the chemical environments of the pockets themselves. As the table below shows, it distinguishes POS from HARD-C in all three splits, with ROC-AUC values of 0.866, 0.891, and 0.916. This provides additional support for the proxy labels from the pocket chemical environments.

PocketFEATURE comparison between POS and HARD-C.

ComparisonTrainingValidationTest
POS vs HARD-C (ROC-AUC)0.8660.8910.916

Note: For both tables, POS is the positive class and HARD-C is the negative class. ROC-AUC is calculated using the SSC or PocketFEATURE similarity score for each pocket pair.

Together, these results show that the proxy labels align with statistical differences in binding contact environments and pocket chemical environments. This supports their use for training and evaluation in our task. However, these results cannot establish that every negative pair is unable to share a ligand. PocketFEATURE comparisons may also be influenced by geometric differences. Matching the geometric similarity distributions cannot fully remove this influence. We therefore treat the chemical labels as proxies supported by evidence and retain these limitations when interpreting their biological meaning.

We have summarized the additional validation in blue in Section 4.1 of the revised manuscript. The methods and complete results in Appendix B.4 (Tables 10 and 11) are also marked in blue.

References

[1] Ehrt C, Brinkjost T, Koch O. A benchmark driven guide to binding site comparison: An exhaustive evaluation using tailor-made data sets (ProSPECCTs). PLoS Computational Biology, 2018, 14(11): e1006483.

[2] Govindaraj RG, Brylinski M. Comparative assessment of strategies to identify similar ligand-binding pockets in proteins. BMC Bioinformatics, 2018, 19: 91.

[3] Liu T, Altman RB. Using Multiple Microenvironments to Find Similar Ligand-Binding Sites: Application to Kinase Inhibitor Binding. PLoS Computational Biology, 2011, 7(12): e1002326.

AuthorsOpenReview ↗

Response to Question 2

We added evaluations on two external retrospective benchmarks, using their original labels.

The first is dataset 7 (P7) of ProSPECCTs [1]. P7 contains 49 query pockets, 1,151 candidate pockets, and 56,399 comparisons, with 115 positive pairs under its original labels. It collects pocket similarity relationships reported in published successful applications. It evaluates the ability to retrieve known similar pockets from a diverse pocket collection.

The second is the Vertex Balanced dataset [2]. It contains 676 pocket pairs, with 338 similar pairs and 338 dissimilar pairs. It evaluates the ability to distinguish similar from dissimilar pocket pairs.

All learning-based methods use weights trained and selected on our internal dataset, then frozen. We perform no training, model selection, or threshold tuning on the external benchmarks. We also retain their original labels. This evaluates transfer without external adaptation. The baselines use the training settings reported in the main comparison. We use three random seeds and report the mean ROC-AUC ± sample standard deviation (n=3).

Zero-shot ROC-AUC on external benchmarks (n=3).

MethodP7: self-pairs retainedP7: self-pairs removedVertex Balanced
GCMatch0.895 ± 0.0160.816 ± 0.0270.919 ± 0.010
OneProt0.734 ± 0.0240.689 ± 0.0200.653 ± 0.016
DeeplyTough0.814 ± 0.0430.677 ± 0.0740.745 ± 0.092
ProFSA0.880 ± 0.0110.791 ± 0.0180.860 ± 0.013
BindSiteS-CNN0.848 ± 0.0090.736 ± 0.0160.742 ± 0.016

As the table above shows, GCMatch achieves the highest mean in all three evaluation settings. It exceeds the strongest baseline in each column by 0.015, 0.025, and 0.059, respectively. We also evaluated the baselines with G/C/Y supervision. The complete results are provided in Appendix I, Table 22, of the revised manuscript. GCMatch retains the highest mean in every column.

The labels of these two benchmarks are not constructed using our combined PPS-align and ECFP4 rule. GCMatch retains its performance advantage without external adaptation. These results show that its learned pocket representations transfer to these two independent evaluations and support its generalization beyond our four-quadrant dataset.

We have added the external evaluation settings in Section 4.2 and the main results in Section 4.3 and Table 2 of the revised manuscript. These additions are marked in blue. The complete settings and results in Appendix I (Table 22) are also marked in blue.

References

[1] Ehrt C, Brinkjost T, Koch O. A benchmark driven guide to binding site comparison: An exhaustive evaluation using tailor-made data sets (ProSPECCTs). PLoS Computational Biology, 2018, 14(11): e1006483.

[2] Eguida M, Rognan D. A computer vision approach to align and compare protein cavities: application to fragment-based drug design. Journal of Medicinal Chemistry, 2020, 63(13): 7127–7142.

Reviewer_zqo3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GCMatch compares protein pockets under a definition of similarity that requires agreement in both cavity geometry and chemical environment. It represents each pocket using cavity points extracted with fpocket and rule-based chemical functional points, encodes them jointly with a GVP-GNN, and retains node-level representations for matching. Geometric points and chemical points are compared separately using bidirectional MaxSim, with chemical matching restricted to corresponding functional types. The two scores are multiplied and trained using both branch-level and final labels.

The paper also constructs a four-quadrant benchmark from PLINDER. Geometric labels come from PPS-align, while chemical labels use bound-ligand fingerprint similarity as a proxy. GCMatch achieves a Full-AUC of 0.920, compared with 0.869 for the strongest baseline. Ablations examine graph connectivity, representation granularity, supervision, and score fusion.

Critical strengths

The benchmark separates two kinds of difficult negatives: pairs that agree in geometry but differ in the chemical proxy, and pairs with the opposite relationship. This makes it possible to examine failure cases that an aggregate pocket-similarity score would hide. The homology-based split and distribution controls add value to this construction.

The paper provides a useful controlled comparison of representation granularity. With the same supervision, node-level matching outperforms branch-level and type-level pooling. This supports the choice to retain local pocket information, even though the broader benefit outside this benchmark remains untested.

Critical weaknesses

The paper does not establish what its new definition of similarity adds to the intended applications. The benchmark defines a positive pair as one that passes both a PPS-align criterion and a bound-ligand fingerprint criterion. This is a clear operational definition, but the paper does not establish why it is the appropriate target for pocket retrieval, functional annotation, or drug repurposing. In particular, a geometrically similar pair with dissimilar bound ligands is labeled negative without evidence that the pockets are chemically incompatible or unsuitable for ligand transfer. Such pairs could include cases worth finding in the applications used to motivate the work. My concern is therefore not simply that the chemical label is a proxy. The benchmark changes which pairs count as successful matches, and the experiments evaluate agreement with that change. They do not show whether the new criterion improves a useful downstream decision or primarily favors methods trained to reproduce it. Validation with additional ligand fingerprints does not resolve this issue because it preserves the same source of supervision. As presented, the dataset is better supported as a controlled diagnostic task than as a general benchmark for pocket similarity.

The main comparison does not separate the benefit of the proposed representation from the benefit of matching the benchmark’s label structure. The target is explicitly defined as Y=G×CY=G\times C. GCMatch receives supervision for both GG and CC, and its final score uses the same product structure. This is a reasonable design, but the baselines are not reported with an equivalent use of the component labels. The paper’s own ablation shows that removing branch supervision lowers Full-AUC from 0.920 to 0.883, leaving a much smaller gap over OneProt at 0.869. Since the contribution lies in the combination of explicit cavity points, chemical points, and local matching, the important comparison is whether these choices still help when other encoders have access to the same supervision and scoring setup. The current experiments do not answer that question. The cross-type-edge ablation also changes the number of connections, so it cannot isolate the benefit of the proposed relation modeling. The results show that the complete system works well on its own task, but provide limited evidence about which methodological contribution explains its advantage.

Questions for authors

What use case should determine whether the benchmark’s negative labels are correct? For geometrically similar pairs assigned C=0C=0, can you distinguish cases with incompatible pocket environments from cases that simply have dissimilar observed ligands? Evidence tied to ligand sharing, interaction compatibility, or a clearly specified downstream task would help establish what these negatives mean.

How much of the improvement remains under matched supervision? Can an existing pocket encoder, trained with the same GG, CC, and YY labels and a comparable two-score matching head, approach GCMatch? This comparison would clarify the contribution of the explicit cavity–chemical point representation.

Does the advantage persist under an evaluation whose labels are not constructed from the same PPS-align/ECFP4 conjunction? An existing external benchmark or a relevant retrieval task would help determine whether the method learns useful pocket similarity beyond the proposed labeling rule. I would not require a wet-lab study, but some independent evaluation is important for the broader claims.

4 comments
AuthorsOpenReview ↗

Response to Question 1

Thank you for reviewing our work and raising these questions.

Pocket matching helps select candidate pockets for further study. In drug repurposing, pockets similar to a known drug's binding pocket can suggest new targets for that drug [1]. In function annotation, local structures and key residues can be compared with known functional sites to infer enzyme activity and substrate specificity [2]. We recognize that pockets outside the dual-similarity criterion may still offer useful leads. However, docking and experimental validation require time and resources, so applications need to prioritize reliable candidates. Similarity in shape or chemical environment alone provides a limited basis for prioritization. A pocket similar in both aspects may offer more reliable target leads or more accurate functional clues. We therefore adopt a matching criterion that requires both geometric and chemical similarity.

We added two analyses to test whether the chemical labels reflect differences in contact environments and pocket chemical environments. Both compare POS pairs with HARD-C pairs. Both groups satisfy the same geometric criterion (G=1), while their chemical labels differ. Their geometric-similarity distributions were also matched during sampling to reduce the influence of geometric differences.

First, we used SSC [3] to compare contact environments in observed protein-ligand complexes. As the table below shows, POS has a higher median SSC than HARD-C in all three splits. The AUC exceeds 0.935 in every split. Thus, the groups also differ in their observed binding contacts.

SSC comparison of POS and HARD-C

SplitPOS / HARD-C pairsMedian SSC (POS)Median SSC (HARD-C)SSC ROC-AUC
Train15,000 / 15,0000.5430.1960.952
Validation1,800 / 1,8000.5500.2050.936
Test2,000 / 2,0000.5640.1910.941

Next, we used PocketFEATURE [4] for an independent validation based on the pockets' own chemical environments. This complements SSC, which still relies on observed protein-ligand contacts. As the table below shows, PocketFEATURE distinguishes POS from HARD-C in all three splits, with AUCs of 0.866, 0.891, and 0.916. These results provide additional support from the pocket environment itself.

PocketFEATURE comparison of POS and HARD-C

ComparisonTrainValidationTest
POS vs. HARD-C (ROC-AUC)0.8660.8910.916

Note: Both tables treat POS as the positive class and HARD-C as the negative class. ROC-AUC is computed from the SSC or PocketFEATURE similarity score for each pocket pair. A higher AUC indicates that POS pairs are more likely to receive higher similarity scores than HARD-C pairs.

Together, the analyses show statistical differences between POS and HARD-C in observed contacts and pocket chemical environments. They provide additional support for our chemical proxy labels. These group differences do not establish that every negative pair is unable to share a ligand. PocketFEATURE scores may also be affected by geometric differences. Matching the geometric-similarity distributions cannot fully eliminate this influence. The analyses therefore support the labels within our dual-similarity matching task, but cannot establish biological incompatibility for individual pairs. We position the four-quadrant dataset as a controlled resource for training and evaluation under a joint geometric and chemical similarity criterion. This task prioritizes candidate reliability; the criterion is not a universal definition of pocket similarity.

We removed the Abstract's claim of a unified evaluation benchmark and narrowed the dataset scope there and in Section 1. We also added the task motivation in Section 1. The revised text is marked in blue. The validation summary in Section 4.1 and the methods and results in Appendix B.4 (Tables 10–11) are also marked in blue.

References

[1] Ehrt et al. A benchmark driven guide to binding site comparison: An exhaustive evaluation using tailor-made data sets (ProSPECCTs). PLoS Computational Biology, 2018.

[2] Amin et al. Prediction and experimental validation of enzyme substrate specificity in protein structures. PNAS, 2013.

[3] Govindaraj and Brylinski. Comparative assessment of strategies to identify similar ligand-binding pockets in proteins. BMC Bioinformatics, 2018.

[4] Liu and Altman. Using Multiple Microenvironments to Find Similar Ligand-Binding Sites: Application to Kinase Inhibitor Binding. PLoS Computational Biology, 2011.

AuthorsOpenReview ↗

Response to Question 2

To examine whether GCMatch's gains come from its representation design or from supervision aligned with the label structure Y=G×CY=G\times C, we added an experiment with matched supervision. For each baseline, we retain its pocket encoder and input representation. Its single pocket vector is passed through two independent linear heads to obtain 64-dimensional geometric and chemical representations. For each pocket pair, cosine similarity between normalized representations is followed by ReLU to obtain sGs_G and sCs_C. The final score is sY=sG⋅sCs_Y=s_G\cdot s_C. Training uses the same pairs, G/C/Y labels, and equally weighted loss terms as GCMatch:

L=BCE(sG,G)+BCE(sC,C)+BCE(sG⋅sC,Y).L=\mathrm{BCE}(s_G,G)+\mathrm{BCE}(s_C,C)+\mathrm{BCE}(s_G\cdot s_C,Y).

All learning-based methods use three random seeds. The table below reports the mean Full-AUC and sample standard deviation (n=3).

Test Full-AUC of baselines before and after adding G/C/Y supervision (n=3)

MethodOriginal training settingWith G/C/Y supervision
OneProt0.873 ± 0.0070.875 ± 0.002
DeeplyTough0.831 ± 0.0130.829 ± 0.005
ProFSA0.859 ± 0.0020.878 ± 0.001
BindSiteS-CNN0.775 ± 0.0140.818 ± 0.018
GCMatch (G/C/Y supervision, our setting)0.921 ± 0.009—
GCMatch (Y-only supervision)0.882 ± 0.010—

With G/C/Y supervision, the Full-AUC of BindSiteS-CNN and ProFSA increases by 0.043 and 0.019, respectively. OneProt and DeeplyTough change by +0.002 and −0.002, within their respective seed standard deviations. The two strongest baselines, ProFSA and OneProt, reach 0.878 and 0.875. They remain about 0.043–0.046 below GCMatch's 0.921. These gaps exceed the respective seed standard deviations. Thus, GCMatch retains a clear advantage when the baselines receive the same supervision.

We have added the main findings in blue in Section 4.3 of the revised manuscript. The experimental setup and full results in Appendix H (Table 21) are also marked in blue.

During the additional experiments with multiple seeds, we found that the original ProFSA and DeeplyTough training runs had stopped at epoch 20. The original manuscript therefore reported results from incomplete training runs. We have now completed the planned 50 epochs and corrected the results. The results reported in this response and the revised manuscript are the corrected results from three seeds. We apologize for this oversight.

AuthorsOpenReview ↗

Response to Question 3

We added evaluations on two external retrospective benchmarks.

The first is dataset 7 of ProSPECCTs [1], referred to as P7. It contains 49 query pockets, 1,151 candidate pockets, and 56,399 comparisons, including 115 positive pairs. P7 collects pocket similarities reported in published successful applications. It evaluates the retrieval of known similar pockets from a diverse pocket library.

The second is Vertex Balanced [2]. It contains 676 pocket pairs, with 338 similar pairs and 338 dissimilar pairs. It evaluates the ability to distinguish similar from dissimilar pockets.

All models use weights trained and selected on our internal dataset, then frozen for external evaluation. We perform no training, model selection, or threshold tuning on the external benchmarks and retain their original labels. This evaluates zero-shot transfer without external adaptation. All learning-based methods use three random seeds. The table below reports the mean ROC-AUC and sample standard deviation (n=3). We evaluate baselines under both their original training settings and G/C/Y supervision. Appendix H of the revised manuscript describes the latter setup.

Zero-shot ROC-AUC on external benchmarks (mean ± sample standard deviation, n=3)

MethodTraining settingP7: self-pairs keptP7: self-pairs removedVertex Balanced
OneProtOriginal0.734 ± 0.0240.689 ± 0.0200.653 ± 0.016
DeeplyToughOriginal0.814 ± 0.0430.677 ± 0.0740.745 ± 0.092
ProFSAOriginal0.880 ± 0.0110.791 ± 0.0180.860 ± 0.013
BindSiteS-CNNOriginal0.848 ± 0.0090.736 ± 0.0160.742 ± 0.016
OneProtWith G/C/Y supervision0.799 ± 0.0030.723 ± 0.0060.787 ± 0.009
DeeplyToughWith G/C/Y supervision0.855 ± 0.0250.748 ± 0.0440.750 ± 0.017
ProFSAWith G/C/Y supervision0.883 ± 0.0000.768 ± 0.0010.800 ± 0.001
BindSiteS-CNNWith G/C/Y supervision0.876 ± 0.0050.784 ± 0.0100.828 ± 0.010
GCMatchG/C/Y supervision (our setting)0.895 ± 0.0160.816 ± 0.0270.919 ± 0.010

As the table above shows, GCMatch achieves the highest mean ROC-AUC in all three evaluations. It exceeds the strongest baseline in each column by 0.012, 0.025, and 0.059, respectively. Neither benchmark derives its labels from the combined PPS-align and ECFP4 rules. GCMatch ranks first without any external adaptation. These results show that its advantage extends beyond our four-quadrant dataset to independent pocket-similarity evaluations.

We have added the external evaluation setup in Section 4.2 and the main results in Section 4.3 and Table 2 of the revised manuscript. These additions are marked in blue. The full setup and results in Appendix I (Table 22) are also marked in blue.

References

[1] Ehrt et al. A benchmark driven guide to binding site comparison: An exhaustive evaluation using tailor-made data sets (ProSPECCTs). PLoS Computational Biology, 2018.

[2] Eguida and Rognan. A computer vision approach to align and compare protein cavities: application to fragment-based drug design. Journal of Medicinal Chemistry, 2020.

AuthorsOpenReview ↗

Response to Additional Concern

We added an edge-count control to address your concern that removing cross-type edges also reduces the number of connections. For each node, we replace the removed cross-type edges with the same number of edges to the nearest unconnected same-type neighbors. This preserves each node's degree relative to the full model. The table below shows the results.

Test Full-AUC under different graph connections (mean ± sample standard deviation, n=3)

ConfigurationGraph connectionsFull-AUC
Full GCMatchOriginal cross-type and same-type edges0.921 ± 0.009
Edge removalCross-type edges removed without replacement0.737 ± 0.020
Edge-count controlCross-type edges replaced with the same number of nearest same-type neighbor edges per node0.777 ± 0.045

After restoring the edge count, Full-AUC remains about 0.144 below the full model. This indicates that the performance drop mainly comes from the loss of cross-type relations rather than the reduced number of connections.

We have added this control experiment in blue in Section 4.4 of the revised manuscript. The results over three seeds in Appendix G (Table 20) are also marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.