- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes to select a Leiden community using classifier confidence for downstream graph classification. The paper reports competitive OOD performance across three datasets under covariate and concept shifts
Critical strengths
The method is straightforward
The paper shows that retaining the full graph reduces information loss
Experimental analysis is broad
Critical weaknesses
Contrary to Section 3.3, stable does not imply stable subgraph-label relationships. Thus, a training-predictive community can be misleading at test time
The paper's own unweighted model scores 74.53 on HIV-covariate (GraphSteer: 74.52), and 92.62 on Motif-concept (GraphSteer: 92.77). Comparisons against much weaker ERM results do not isolate the contribution of reweighting
The pruning baseline removes both nodes and edges, while plain Leiden receives weights tuned for the proposed variant. These comparisons do not cleanly establish the benefits of soft weighting
The authors do not provide code for reproducing the reported results. More concerningly, the Reproducibility Statement (lines 503) states that "Code, configuration files, random seeds, and preprocessing details should be included in the supplementary material or an anonymous repository," which reads like an unedited placeholder
Questions for authors
Please see above
Stability of subgraph-label relationships
We agree that stable P(Y∣G) under covariate shift does not necessarily imply stable subgraph-label relationships. To evaluate this question, we assessed subgraph selection by measuring alignment with known task-relevant structures across splits:
Table 1. Selected-subgraph alignment (%).
CMNIST: selected community contains foreground; Motif: selected community meets the causal-subgraph criterion.
For covariate shift, Motif alignment remains high across splits (98.3%–100%), while CMNIST reaches 100% on OOD splits despite lower ID alignment. Under concept shift, alignment is high on CMNIST (88.6%–94.8%) but lower on Motif (59%–64.2%). These results provide evidence that selection can transfer to unseen data.
We will revise Section 3.3 as follows:
Node-preserving pruning controls
We agree that deleting nodes confounded the previous pruning comparison. We added (1,0,0) (intra-subgraph edges only) and (1,1,0) (intra-subgraph plus boundary edges); both retain all nodes and node features, including isolated nodes. We also report unweighted (1,1,1) and independently tuned Original Leiden.
Table 2. GOOD-CMNIST color/concept ablation (accuracy in %, mean ± SD).
Subgraph only additionally removes external nodes. Original Leiden and GraphSteer select scores and weights independently under the same validation protocol.
GraphSteer exceeds node-preserving Outside-Edge Pruning by 11.91 pp (ID-Val selection) and 10.34 pp (OOD-Val selection), supporting soft weighting over binary removal.
Comparable unweighted results
We acknowledge that the unweighted configuration (1,1,1) achieves comparable performance to GraphSteer on GOOD-HIV covariate (74.53% vs. 74.52%) and GOOD-Motif concept (92.6% vs. 92.77%).
While soft edge reweighting does not always provide substantial gains, it maintains competitive performance without meaningful degradation. As shown in Table 1, it also provides substantial improvements over binary edge pruning under challenging distribution shifts.
Original Leiden tuning
We independently tuned the confidence scores and edge weights for Original Leiden and Node-Feature-Aware Leiden using the same validation protocol. GraphSteer outperforms Original Leiden by 10.56 pp under ID-Val selection (51.34% vs. 40.78%) and 9.23 pp under OOD-Val selection (52.23% vs. 43.00%), demonstrating the benefit of feature-aware partitioning.
Reproducibility
We apologize for the incomplete reproducibility statement. We will revise it to clarify the preprocessing, backbone architectures, hyperparameters, random seeds, and model-selection procedures. If the paper is accepted, we will provide the complete source code, configuration files, and reproducibility details in the final version.