- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes a novel hierarchical pooling operation for graph neural networks, Graph-Dependent Pooling (GDPool) that aims to automatically learn a pooling granularity for each graph using self-attention and soft node assignment. To reach better agreement across multiple pooling layers, GDPool uses hierarchical supervision, applying the pooling objective to the cumulative assignment across pooling levels, and identifies anchors of important nodes in the original graph that define the pooling assignment and the number of nodes in the pooled graph. Performance is evaluated on 4 graph regression tasks compared to three baseline methods, showing that GDPool reaches superior or comparable performance to baseline models.
Critical strengths
Evaluating and learning the best granularity for each graph across multiple pooling levels is of interest for graph pooling overall and the work presents a novel perspective on not just adaptively selecting an appropriate pooling ratio for each graph, but also optimising the hierarchical pooling jointly across multiple pooling layers. Overall, the execution is convincing and the results are easy to follow and appear to be promising.
Critical weaknesses
Comparison with a wider set of baseline methods. It would be useful to compare with a wider range of pooling layers, for example as implemented in the tgp (Torch Geometric Pool), specifically to compare with nontrainable pooling layers using different pooling mechanisms or with further adaptive pooling layers, such as ASAGPool. As it stands, GDPool is mainly compared to dense attention-based pooling methods, such as DiffPool. Further, as suggested below, it would be useful to also compare GDPool to using no pooling layer, or to randomised ablations, to assess which components of GDPool influence or improve performance.
Tuning the coarsening levels of the baseline methods. Further, to ensure consistence, it might be fair to investigate the granularity GDPool selects per graph, and tune the pooling layers it is compared to coarsen graphs to the same or higher numbers of nodes, so that the performance differences are not just driven by the fact that GDP learns consistently lower pooling ratios, e.g. uses less extreme coarsening. For example, if other pooling methods use a pooling ratio of 0.5 for each graph, but GDPool generally uses 0.8, this could already cause substantial performance differences. Based on Figure 3 this does not necessarily seem to be the case for pooling-ratio based methods on hMOF for the first pooling layer. But for fixed cluster-based methods, it might be worth increasing the number of clusters to evaluate whether tuning the number of clusters selected, the baseline methods could not also reach more comparable performance to GDPool. Similarly, it would be useful to repeat such ablations for more datasets and for the 2nd and 3rd pooling layers as well, to track whether the degree of coarsening remains broadly comparable to the baselines or not.
Modularity depending on the pooling mechanism. Could you report Table 2 for a wider range of pooling methods, e.g. also including edge pooling or sparse pooling operations? There is reason to believe that hard rather than soft node assignment methods might retain higher modularity.
Limited number of evaluated tasks. The main experiments evaluate 4 graph-level regression tasks, which by themselves show only a limited number of examples. It would be useful to compare across a wider range of tasks, for example graph-level classification tasks, which are commonly reported on. Especially datasets with larger graph sizes would be of interest.
Computational costs as a major limitation. GDPool itself seems to be relatively computationally expensive as reported in Table 3. It is known that as dense pooling methods, DiffPool and MinCutPool already scale relatively badly to large graphs, with GDPool reaching worse runtimes and higher memory usage. This is a clear limitation of the proposed method, likely ensuring that it will not scale to very large graphs.
Minor comments:
- It would be useful to define the lower-level assignment S(l) in line 151 in terms of the pooling operation, especially for readers unfamiliar with graph pooling.
- Conceptually, why would you think that layers should align? Is there one correct pooling objective per graph? Or would it not also still be possible to get a sensible pooling overall through conflicting objectives?
Questions for authors
- Could you evaluate a random version of your Empool operator? Similar to the ablations reported in “Rethinking pooling in graph neural networks” by Mesquita et al. (2020), it would be interesting to evaluate a randomised version of your algorithm that instead of learning the node assignment, for example uses a random selection of anchor nodes and potentially also a random node assignment. Such ablations could reveal which components of GDPool influence performance.
- Could you compare the performance of GDPool to the same GFFN architecture without using any pooling layers? That is, repeat the experiments without any pooling to see whether the proposed pooling operation increases or decreases performance.
- Could you report further ablations on the choice of pooling ratio or cluster numbers for the baseline pooling methods compared across your experiments, and how they match your own method’s node selection?
2 comments
Update to our reply. Tables 17 and 18 now cover the same six datasets. This revises three statements above:
- Following reviewer jejw's suggestion, REDDIT-MULTI-12K is removed from the accuracy tables, so Table 7 reports Peptides-func only, on which GDPool remains the best (AP 0.599 against 0.488).
- In Table 18, GDPool has the best mean on four of the six datasets; PageRank anchors are slightly better on ZINC and random assignment on Peptides-func, both within one standard deviation.
Our common comment summarizes all changes in the revision.
We thank the reviewer for the detailed review. We understand "Empool" in the questions to refer to GDPool.
MAE on hMOF CO₂@0.1, mean ± std over three seeds (Table 16). Default: K = 32/16/8 or r = 0.5 at every level. Matched: K = 20/10/6 or r = 0.15/0.5/0.5.
We hope that our responses and the revisions address the reviewer's concerns, and we would appreciate it if the reviewer could reconsider the evaluation.