- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes GraphDETR, which frames subgraph detection as a set prediction task. A variety of GNN encoders (GCN, GIN, GatedGCN, GraphGPS, and NeuralWalker) are tested to produce node embeddings. A DETR/SAM-style two-way transformer decoder with learnable queries then predicts (class, node-mask) pairs, and is trained with Hungarian matching and a graph-cut regularizer. Then there is an evaluation on molecular functional-group detection (ChEMBL, 95 classes) and on synthetic benchmarks (cactus, cliques, ZINC12k, Mol-Reddit), with a comparison against VF2 on runtime.
Critical strengths
- The set-prediction formulation of subgraph detection is clear. The analogy to object detection is natural, and the pipeline is end-to-end with no NMS or combinatorial search at inference.
- The evaluation is broad. They evaluate on a large scaffold-split ChEMBL dataset, several synthetic families, exact and fuzzy matching, 5 seeds for the main tables, and runtime measurements.
- Subgraph detection is a useful test for comparing the expressivity of graph encoders. The finding that global or walk-based encoders outperform local MPNNs is informative.
- The ablations (loss terms, decoder capacity, and NeuralWalker settings), the appendix details, and the limitations section are useful.
Critical weaknesses
-
The technical/methodological noveltyis limited. The model uses DETR plus the SAM two-way decoder plus off-the-shelf encoders. The only graph-specific component is the cut penalty, which is a standard graph-cut/total-variation-style regulariser.
-
Insufficient baselines. The encoder comparison (section 4.1) compares backbones only inside the framework of the authors, so it can't show the gain of the framework itself. As far as I saw, the only external learned baseline is NeuroMatch, which gets one paragraph in the main text and reaches under 1% AP100. Its adaptation is only described in Appendix D and adds several components that are not part of NeuroMatch (a canonical exemplar per class, calibrated thresholds, region growing for masks, and duplicate suppression). The authors mention these choices affect the result, so it is not clear to me whether the near-zero score reflects NeuroMatch or the pipeline built around it. For me, missing comparisons are:
- node-level tagging plus connected components, or instance embeddings plus clustering.
- a standard DETR or Mask2Former-style head without the two-way decoder.
- other neural subgraph matching and counting methods.
- for molecules, the rule-based detector that produced the labels (Peter Ertl's algorithm). It is exact by construction, so the relevant comparison is runtime, which is not reported.
-
The practical value for the main task is unclear. Functional-group detection in molecules is already well served by rule-based tools that are exact and fast. The motivating benefits (differentiability, retrosynthesis, and property prediction) are not tested/evaluated, and the paper does not say in which setting a learned detector would be well-suited.
-
Design claims that lack supporting experiments.
- "Using H_ref rather than the raw encoder output H is critical". Why is it critical? I cannot find results showing this.
- The two-way decoder is designed for overlapping instances, does a benchmark contain them?
- The symmetry-breaking argument for random walks is never shown with an experiment.
-
Tables 2–4 report no standard deviations and many conclusions rest on 1-2 point differences. NeuralWalker is also the only encoder with two λ_mask settings and has more parameters (7.4M versus 3.9 to 6.6M). There's no parameter-matched or equally tuned comparison.
-
The claim that GraphDETR is "the first model to jointly predict all instances of a fixed pattern set in a single differentiable forward pass" is strong. There are models like query-based instance segmentation on point clouds and substructure-counting GNNs that are close analogues and should be discussed. The closed vocabulary setting also limits applicability, since the pattern set must be fixed and a model retrained for each new one.
Questions for authors
- How does GraphDETR compare to simple non-query-based alternatives built on the same encoders, namely node-level multi-label tagging with connected components, or instance embeddings with clustering? How does it compare to a standard one-way DETR decoder?
- Can you ablate the use of H_ref versus the raw encoder output H in the mask head? Can you also report what fraction of instances overlap in each benchmark, with performance on graphs with and without overlaps? Finally, can the random-walk symmetry-breaking argument be tested directly, for example on molecules with automorphic nodes (such as CF3 groups), or by adding random node features to an MPNN?
- On ChEMBL, in which setting would a learned detector be preferable to a rule-based functional-group detector, given that it is less exact? Can you show a downstream benefit (e.g. property prediction or retrosynthesis) or a case where rule-based labels are unavailable?
- Can you report standard deviations for the ablation tables (Tables 2-4), and confirm that λ_mask and other hyperparameters were tuned equally for all encoders, ideally with a parameter-matched NeuralWalker vs. GraphGPS comparison?
5 comments
Part 2/3
Is H_ref critical compared with raw H?
We ran the suggested ablation. Table R1 above compares masks computed from H_ref and raw H. Raw H decreases ExactMatch significantly on ChEMBL12k / Cliques100. Thus H_ref does help on two of the three datasets. We report these results in Appendix G.4 and revised the claim in Section 3.3 accordingly.
Overlap prevalence and performance on graphs with/without overlap.
Yes, several benchmarks contain shared-node instances. The following test-split measurements distinguish shared-node overlap from the broader case of instances that share a node or are connected by an edge. Percentages refer to instances, not graphs.
| Benchmark | Shared-node overlap (%) | Overlap or edge adjacency (%) |
|---|---|---|
| Cactus40 | 86.9 | 97.8 |
| Cactus200 | 56.3 | 79.3 |
| Cactus1000 | 32.0 | 49.9 |
| Cliques100 | 0.0 | 38.8 |
| Cliques200 | 0.0 | 37.6 |
| Cliques400 | 0.0 | 37.0 |
| ZINC12k | 85.5 | 89.2 |
| Cliques-fuzzy | 0.0 | 19.7 |
| Cactus-fuzzy | 58.1 | 76.8 |
| Mol-Reddit | 0.0 | 0.0 |
| Chembl12k | 0.0 | 10.2 |
Cactus40 is the only dataset in Table R1 with shared-node overlap, so we split its test graphs into those with and without shared nodes (AP100 / ExactMatch). These numbers come from a separate evaluation pass and differ slightly from Table R1.
| Detector | No shared-node overlap (9 graphs) | Shared-node overlap (1,191 graphs) |
|---|---|---|
| GraphDETR | 99.8 / 88.9 | 97.2 / 86.7 |
| Tagging + CC | 75.9 / 51.9 | 61.0 / 33.2 |
| Instance embeddings + clustering | 100.0 / 100.0 | 67.4 / 40.5 |
| One-way decoder | 97.1 / 70.4 | 94.1 / 36.6 |
| Raw-H mask head | 100.0 / 100.0 | 98.5 / 87.4 |
Note that only 9 of the 1,200 Cactus40 test graphs contain no shared node, so the left column should be read with caution. The comparison between graphs with and without overlap is best seen across datasets. ChEMBL12k and Cliques100 contain no shared-node overlap, while in Cactus40 87% of instances share a node. Table R1 shows that instance embeddings + clustering is competitive on the former but fails on the latter, whereas GraphDETR performs well on both.
We now include these results in Appendix E.3 in the revision.
Evidence for symmetry breaking, including automorphic nodes and random features.
Following the suggestion, we evaluated molecules with automorphic nodes. On ChEMBL12k, 166 of 1,000 test molecules contain several instances of the same functional group whose atoms are automorphic to each other, such as the three fluorines of CF3. On this subset, GIN reaches only 7.2% ExactMatch, compared to 79.7% on the remaining molecules, whereas NeuralWalker degrades much less (69.9% vs 83.4%). This is consistent with sampled walks breaking the symmetry between automorphic nodes, which deterministic encoders cannot, even with RWSE. The subsets may, however, differ in other respects, so this does not establish a causal explanation. Fig. 7 (Success 12 and 24) shows CF3 groups recovered by NeuralWalker trained on full ChEMBL. We include these results in Appendix G.2 in the revision.
We note that our goal is not to exhaustively identify the best encoder for subgraph detection, but rather compare existing representative graph encoders under our unified setting. Our subgraph detection benchmark could be a valuable testbed for expressive GNNs, and we leave testing with random feature GNNs for future work.
Standard deviations for Tables 2–4 and claims based on small differences.
Running full five-seed experiments for every configuration in Tables 3–5 would require thousands of GPU-hours, which is beyond the compute budget available during the response period. The current single-run ablations provide an initial picture of how performance changes with each hyperparameter and help identify trends that guide the model configuration, which serves as justifications for our hyperparameter choices. They do not establish the statistical significance of individual differences or a definitive ranking of nearby settings. We agree that these ablations are exploratory and do not support conclusions from small differences. Variability measured in other experiments cannot substitute for standard deviations for these configurations. For all new results, we now report mean and standard deviation over multiple seeds (Tables 2 and 18 of the revised paper).
Part 3/3
Were λ_mask and the other hyperparameters tuned equally for all encoders?
Yes. Every encoder was swept over the same grid on ChEMBL12k: λ_mask = α in {2, 4, 8}, background weight in {0.1, 0.5}, λ_cut in {0, 0.001, 0.01}, with the same architecture setting (5 layers, hidden 256, RWSE 20). Table 1 lists two NeuralWalker settings because the two best λ_mask values rank very differently, as one wins on AP100 and the other on ExactMatch. The other encoders rank consistently across metrics, so one setting suffices. We keep both NeuralWalker rows to show this, rather than pick one metric. Table 1 is a study of how encoders behave on subgraph detection, not necessarily a search for an encoder that is best on every metric. We now state this explicitly in Appendix F.2 of the revised paper.
Parameter-matched NeuralWalker–GraphGPS comparison.
All encoders use the same number of layers and hidden dimension, so their parameter counts differ only through the architecture itself. In fact, several observations indicate that the ranking is not driven by model size:
- GraphGPS is slightly smaller than NeuralWalker (6.6M vs 7.4M), yet outperforms it on Cliques100/200/400 and on Cactus40 ExactMatch.
- The smallest encoders, GCN (3.9M) and GIN (4.9M), still reach 90.9 and 93.3 AP100 on Cliques100.
- The gaps to message-passing encoders on Cactus200 and Cactus1000 (50 and 37 AP100) are arguably too large to stem from at most 3.5M parameters.
A fully parameter-matched comparison was unfortunately not feasible within the response period. Note that our goal is not to identify the best encoder for subgraph detection, but rather compare existing representative graph encoders under our unified setting. Our subgraph detection benchmark could be a valuable testbed for expressive GNNs, and we leave a more comprehensive comparison for future work.
Query-based segmentation and counting methods as related work.
Query-based segmentation methods are architectural precedents for GraphDETR, but operate on pixels or points with spatial coordinates, whereas graph instances must be separated by their connectivity alone. Substructure-counting GNNs predict how often a pattern occurs, but not where. We thank the reviewer for these suggestions and extended the related work section accordingly.
Closed vocabulary and retraining for new patterns.
A fixed pattern library is a standard use case for subgraph detection, for example functional groups or network motifs, which are defined once and then searched in various graphs. GraphDETR learns such a library once and predicts all instances of all patterns jointly, so retraining is only needed when the library itself changes. This mirrors object detection, where closed-vocabulary models such as DETR preceded open-vocabulary extensions. We acknowledge this limitation and discuss such an extension of GraphDETR in Appendix A.
Part 1/3
We thank the reviewer for the detailed review and address each point below.
We agree that GraphDETR builds on several established components, and we do not claim novelty for these components individually. We believe, however, that our work should not be judged only by the novelty of its components, but by the problem it solves. Finding all occurrences of given patterns is a fundamental task in graph analysis, for example in motif analysis and frequent pattern mining. Classical solvers such as VF2 enumerate occurrences by combinatorial search, but only for exact patterns and one pattern at a time. Existing neural approaches, in contrast, answer only whether a pattern occurs, or recover a single node mapping. To our knowledge, GraphDETR is the first neural model to enumerate all occurrences of given patterns in a graph. We would also argue that it is not a straightforward combination of existing components. Formulating the task as direct set prediction and identifying which components are suitable for it is part of our contribution, and our ablations show that these choices are crucial (Appendix G.4). In addition, we provide an evaluation setup with exact, fuzzy and overlapping instances, which allows future methods to be compared on this task. We revised the related work section accordingly.
The NeuroMatch numbers must indeed be interpreted in light of the adaptation. Since NeuroMatch natively answers whether a query occurs around a given anchor, enumerating all instances requires significant changes (App. D). It should therefore not be considered a direct competitor. We include it as the most established neural matcher. Our results show that a matcher answering only whether a pattern occurs does not directly extend to enumerating its instances. We include a brief discussion in Appendix D.1 in the revision.
We ran additional experiments to identify the cause of the low scores of NeuroMatch. With its original local k-hop neighbourhoods instead of the full graph, NeuroMatch reliably detects where patterns occur (AUC increased from 0.5 up to 1.00), yet completely fails to recover the instances themselves (AP100 at or below 1%). Even when the correct anchor nodes are given, building the instances from them reaches only 39.2 / 1.2 / 0.0 AP100 on ChEMBL12k / Cactus40 / Cliques100. The bottleneck is thus turning detected occurrences into instance sets.
We agree that comparing encoders within our framework does not show the benefit of the framework itself. We therefore ran the proposed comparisons, using the same encoder and training setup (see Table R1 in general response). We now report these results in Section 4.2 and Appendix G.4 of the revised paper.
1a. Tagging + connected components is strong when same-class instances are separated (best AP100 on ChEMBL12k, 98.8 vs 93.7), but merges adjacent same-class instances into one component (Cactus40 ExactMatch 33.6 vs 84.0).
1b. Instance embeddings + clustering is strong when instances are node-disjoint (ChEMBL12k ExactMatch 94.8 vs 83.4), but assigns each node to a single instance and cannot represent shared nodes (Cactus40 ExactMatch 41.2 vs 84.0).
2. A one-way DETR-style decoder lowers AP100 and ExactMatch on all three datasets (Cactus40 ExactMatch 36.6 vs 84.0). Since it uses our set-prediction formulation, we consider it an ablation rather than a baseline.
3. Other neural matching and counting methods answer whether a pattern occurs, return a single node mapping (Sub-GMN, xNeuSM), or predict counts. None enumerates instances without substantial adaptation, as discussed for NeuroMatch above. We are happy to include a specific method if the reviewer has one in mind.
We would like to clarify that functional-group prediction is meant as a case study, not the main task of the paper. ChEMBL provides a large set of real molecules, and Ertl's algorithm labels their functional groups exactly, which makes it a well-suited testbed for our method. We do not aim to replace this algorithm itself. We clarify this in the revision.
Ertl's algorithm is a rule set designed specifically for functional groups. GraphDETR instead learns any given pattern library from labelled data, and therefore applies where hand-crafted tools fall short, such as (i) fuzzy patterns, where exact methods do not apply, (ii) large graphs, where exact search becomes impractical, and (iii) pipelines that require a differentiable detector. The downstream applications named in App. B, retrosynthesis and property prediction, were meant as motivation rather than tested claims.