- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies membership inference attacks (MIAs) against Simplicial Complex Neural Networks (SCNs), focusing on privacy leakage arising from higher-order topological structures. The authors propose a structure-aware black-box attack (SA-BB) that exploits local prediction patterns to infer training membership. They further propose Dynamic Topological Pruning and Injection (DTPI), a defense that modifies topological relations during training to mitigate privacy leakage. This paper evaluates the effectiveness of SA-BB and the privacy–utility trade-offs achieved by experiments across different datasets, model architectures, and attack and defense baselines.
Critical strengths
-
This paper proposes the SA-BB attack, which exploits local prediction patterns to infer training membership without requiring access to the target model's internal parameters. Experimental results demonstrate its effectiveness compared with some existing membership inference baselines.
-
This paper proposes DTPI defense, which dynamically modifies topological structures to mitigate membership leakage while preserving model utility. Experiments demonstrate favorable privacy–utility trade-offs across different attacks, and ablation studies validate the contributions of its key components.
-
This paper is well-organized and easy to follow.
Critical weaknesses
-
This paper evaluates several membership inference attacks and defenses; however, the comparisons mainly rely on conventional baselines, leaving the advantages of the proposed methods insufficiently validated. For attack evaluation, stronger approaches such as LiRA [1], RMIA [2], and recent topology-aware attacks [3] are not considered. Similarly, the defense evaluation lacks comparisons with established methods such as SELENA [4] and more directly related topology-based defenses [5]. Including them would provide a more comprehensive and convincing evaluation of SA-BB and DTPI.
-
The proposed SA-BB attack shows limited advantages over existing methods at low FPR. On SEMANTIC, SA-BB achieves a TPR of only 2.14% at 1% FPR, showing almost no improvement over the conventional NN attack (2.01%). Similarly, the proposed DTPI defense still suffers from substantial membership leakage. On CITESEER, SIA achieves a TPR of 41.54% at 1% FPR even after applying DTPI, indicating that considerable membership information remains exposed. These results raise concerns about the effectiveness of both proposed methods.
-
The core design of SA-BB relies on relatively straightforward techniques, including prediction confidence, neighborhood-based statistics, and a logistic regression classifier trained on shadow models. Previous studies, such as SIA and SMIA, have already explored exploiting structural information for membership inference in graph neural networks. While SA-BB extends this direction to simplicial complexes under a different access setting, its methodological novelty beyond this adaptation remains unclear. Moreover, Table 10 shows that degree-matched neighbors outperform topology-based neighbors on CITESEER, raising further questions about whether the improvement primarily comes from simplex-specific topological information or more general structural statistics.
-
The core design of SA-BB relies on relatively conventional techniques, including prediction confidence, neighborhood statistics, and a logistic regression classifier trained on shadow models. Previous studies have already explored structural inference attacks on GNNs [6], while SMIA [7] specifically exploits model predictions and shadow training to infer higher-order subgraph structures. Although SA-BB extends this research direction to simplicial complexes, its technical novelty beyond this adaptation remains unclear. Furthermore, Table 10 shows that degree-matched neighbors without shared boundary faces outperform topology-based neighbors on CITESEER, raising questions about the unique contribution of the proposed face-sharing neighborhood design.
[1] "Membership Inference Attacks From First Principles", Carlini et al., IEEE S&P 2022.
[2] "Low-Cost High-Power Membership Inference Attacks", Zarifzadeh et al., ICML 2024.
[3] "Attention-Based Membership Inference Attacks on Graph Neural Network Through Topological Features", Guan et al., IEEE TDSC 2025.
[4] "Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture", Tang et al., USENIX Security 2022.
[5] "Topology Modification against Membership Inference Attack in Graph Neural Networks", Guan et al., Knowledge-Based Systems, 2024.
[6] "Inference Attacks Against Graph Neural Networks", Zhang et al., USENIX Security 2022.
[7] "Subgraph Structure Membership Inference Attacks against Graph Neural Networks", Wang et al., PoPETs 2024.
Questions for authors
-
How do the proposed SA-BB attack and DTPI defense compare with more recent and stronger membership inference attack and defense methods? Please refer to Weakness 1 for details.
-
What are the key technical differences between SA-BB and existing structure-aware membership inference attacks? Please further clarify the methodological novelty of SA-BB and provide additional evidence supporting the benefits of its simplex-specific design.
4 comments
Response to Reviewer Tcuy (Part 3)
W3, W4 & Q2. Differences from SIA [6] and SMIA [7], classifier choice, and Table 10
(Since W3 and W4 raise the same points on [6, 7] and Table 10, we address them together.)
Technical differences from SIA [6] and SMIA [7]. Table 3 summarizes the distinction. SIA [6] and SMIA [7] target node-classification GNNs whose API returns multi-class posterior vectors per node; in particular, SMIA infers subgraph structure from pairwise cosine-similarity rankings across multi-dimensional node posteriors (or embeddings). In -simplex prediction, the black-box API returns only one scalar probability per candidate simplex, so pairwise node-posterior similarity matrices cannot be constructed from black-box outputs. Evaluating SIA and SMIA on simplex prediction in submitted Table 7 therefore required granting them white-box access to internal edge or node embeddings.
Table 3. Comparison with structure-aware GNN attacks.
| Method | Target Model & Task | Model Access & Output Granularity | Structural Signal |
|---|---|---|---|
| SIA [6] | GNN, node classification | Node multi-class posteriors (white-box edge embeddings in Table 7) | Pairwise node-output / embedding similarity for 1-skeleton links |
| SMIA [7] | GNN, node classification | Node multi-class posteriors on subgraph (white-box node embeddings in Table 7) | Pairwise cosine-similarity rankings across node posteriors |
| Guan et al. [3] | GNN, node classification | Node multi-class posteriors of node + 1/2-hop neighbors | Attention aggregation over node-posterior neighborhoods |
| SA-BB (ours) | SCN, -simplex prediction | Black-box simplex API: one scalar per candidate simplex | Scalar calibration against public -face / coface candidates |
Classifier choice. Holding standardized logistic regression and balanced shadow splits fixed across NN, Loss, and SA-BB (with a fixed neighbor budget cap for SA-BB; Appendix D.2 and E.5) is an intentional control so that gains can be attributed strictly to the topological calibration features rather than classifier capacity. Within the overall paper, SA-BB serves as the black-box counterpart to our Hodge-Laplacian exposure analysis (Proposition 4.2) and cascade-guided defense (DTPI).
Neighbor selection in Table 10. In Appendix E.5, Matched aligns the mean and standard deviation of candidate-triangle coface counts across 's three boundary edges (not 1-skeleton vertex degree). Also, fallback in Section 4.3 triggers only when (" prediction calls" in Appendix E.5 is the budget cap), so sparse CiteSeer queries with use fewer than 20 neighbors under unpadded Topology. Table 4 adds a 1-skeleton Node-degree matched control and Topology + padding (full table in Reviewer 7PWn Table 1).
Table 4. SA-BB neighbor-selection breakdown (, SCCNN; AUC / TPR@5% on Semantic and CiteSeer, AUC on Cora).
| Selection Rule | Semantic () | CiteSeer () | Cora () |
|---|---|---|---|
| Random | 0.700 ± 0.028 / 0.104 | 0.849 ± 0.005 / 0.206 | 0.930 ± 0.012 |
| Node-degree matched (1-skeleton vertex degree) | 0.706 ± 0.026 / 0.081 | 0.875 ± 0.005 / 0.341 | 0.929 ± 0.013 |
| Matched (2-simplex coface count) | 0.702 ± 0.022 / 0.101 | 0.897 ± 0.005 / 0.463 | 0.937 ± 0.011 |
| Topology (face-sharing) | 0.729 ± 0.029 / 0.133 | 0.885 ± 0.010 / 0.377 | 0.950 ± 0.012 |
| Topology + padding | 0.728 ± 0.026 / 0.131 | 0.899 ± 0.005 / 0.486 | 0.946 ± 0.011 |
2-simplex calibration yields SA-BB's strongest results on every dataset: Topology leads on Semantic (0.729 / 0.133) and Cora (0.950), and Topology + padding leads on sparse CiteSeer (0.899 / 0.486, comparable to Matched at 0.897 / 0.463 once partial neighborhoods are padded to ). In contrast, 1-skeleton Node-degree matched trails 2-simplex rules on CiteSeer and Cora (matching Random on Cora) and stays near Random alongside Matched on dense Semantic.
[6] "Inference Attacks Against Graph Neural Networks", Zhang et al., USENIX Security 2022.
[7] "Subgraph Structure Membership Inference Attacks against Graph Neural Networks", Wang et al., PoPETs 2024.
Response to Reviewer Tcuy (Part 2)
W2. Low-FPR behavior on Semantic (2.14%) and SIA on CiteSeer (41.54%)
Low-FPR behavior of SA-BB. Compared with CiteSeer (average degree 2.736, Appendix D.1), Semantic is a dense co-authorship complex (average degree 8.375). Heavy message passing across shared edges smooths local representations, causing high-confidence members and well-connected non-members to overlap in the extreme 1% FPR tail (2.14% TPR for SA-BB vs. 2.01% for NN and 1.43% for Loss in submitted Table 9). At 5% FPR on Semantic (submitted Table 9 and Table 1), local calibration separates these regimes clearly: SA-BB reaches 13.3% TPR, a 43% relative improvement over NN (9.3%) and 47% over Loss (9.1%), outperforming Att-Topo (12.3%) and matching RMIA (13.7%) while achieving the highest overall AUC (0.729). On sparser CiteSeer, where face-sharing propagation creates sharper local contrast around training triangles, SA-BB improves substantially at both 1% FPR (18.4% vs. 6.4% for NN and 5.0% for Loss in submitted Table 9) and 5% FPR (37.7% vs. 14.4% for NN, 33.2% for online LiRA, 32.6% for RMIA, and 41.5% for RMIA + SA-BB in Table 1).
Residual white-box SIA (41.54%) on CiteSeer. Two factors account for this figure. First, as noted in Appendix D.2, SIA is a white-box audit with access to internal edge embeddings and ground-truth task labels, which are unavailable through a prediction API (where DTPI reduces black-box NN TPR@1% to 5.95% in submitted Table 8, and reduces SA-BB and online LiRA TPR@5% to 20.6% and 23.7% in Table 2). Consistent with the utility–exposure coupling in Proposition 4.2, preserving predictive utility on sparse CiteSeer relies on message passing over core training triangles, and empirically internal edge embeddings retain structural correlations unless utility collapses; in submitted Table 8, even DP-SGD at collapsed utility (0.6593 Model AUC) still leaves SIA TPR@1% at 48.98% (vs. 50.67% undefended in Table 7). Second, relative to all evaluated defenses, DTPI provides the strongest mitigation against SIA: in submitted Table 8, DTPI achieves the lowest SIA TPR@1% (41.54%, while all six baselines remain between 46.89% and 51.35%) at the highest Model AUC (0.7807), and in Table 2 at matched utility (0.855 Test Model AUC) it achieves the largest reduction in both SIA AUC (0.927 to 0.864) and TPR@5% (0.621 to 0.463).
Thank you for the detailed response and additional experiments. I appreciate the new comparisons with recent attack and defense methods, as well as the clarifications regarding the differences from existing approaches. These additions address some of my concerns.
However, I still have reservations about the technical novelty of SA-BB, as its core methodology remains largely based on conventional membership inference techniques. In addition, the attack performance at low FPR on SEMANTIC remains less convincing. Specifically, at 1% FPR, SA-BB achieves a TPR of only 2.14%, showing almost no improvement over the conventional NN attack (2.01%). While the results at 5% FPR are better, I consider 5% FPR to be well beyond the low-FPR regime, which, in my view, should focus on FPRs no greater than 1%, or even as low as 0.1%.
Therefore, I will maintain my original rating.
Response to Reviewer Tcuy (Part 1)
We thank the reviewer for the positive comments on SA-BB, DTPI, and the paper's presentation, as well as for suggesting recent attack and defense baselines [1–5] and a closer comparison with [6, 7].
W1 & Q1. Comparison with recent attacks [1–3] and defenses [4–5]
We extend both benchmarks on SCCNN under the 3-target-seed × 3-shadow protocol (balanced 1:1 membership split, mean ± std):
1.Attacks (Table 1). Alongside NN and Loss from submitted Table 9, we evaluate offline/online LiRA [1], RMIA [2], and Att-Topo (adapting [3] from multi-class node posteriors to a scalar simplex API via a shadow attention aggregator over face-sharing neighbor predictions).
2.Defenses (Table 2). Selecting configs at validation Model AUC and reporting held-out Test Model AUC, we compare DTPI with No Defense, SHAP, GEP, SELENA [4] (Split-AI sub-models with strictly isolated simplicial operators plus self-distillation), and Topo-Mod [5] (degree-constrained edge edits rebuilt into a valid complex), retraining all black-box attacks on three identically defended shadows.
Table 1. Undefended black-box attack comparison on SCCNN (NN and Loss from submitted Table 9; remaining rows over 3 target seeds × 3 shadows).
SA-BB achieves the strongest standalone black-box performance on both datasets, leading online LiRA, RMIA, and Att-Topo in AUC on CiteSeer (0.885) and Semantic (0.729) as well as CiteSeer TPR@5% (0.377). Explicit rank/gap calibration outperforms Att-Topo's scalar attention under a single-probability API, and combining SA-BB with RMIA further boosts CiteSeer to 0.891 AUC and 0.415 TPR@5% (0.736 AUC on Semantic).
Table 2. Matched-utility defense comparison on CiteSeer (SCCNN; 3 target seeds × 3 defended shadows; configs selected at validation Model AUC 0.8340±0.01).
DTPI delivers the strongest overall defense at matched utility, retaining 0.855 Test Model AUC while achieving the lowest AUC and TPR@5% against SA-BB, online LiRA, and white-box SIA. Topo-Mod lowers RMIA further (0.768 / 0.250 vs. 0.794 / 0.268) at the cost of lower utility (0.842) and weaker protection on the remaining three attacks, whereas SHAP, GEP, and SELENA (24$\times$ slower) fail to curb SIA (>0.92 AUC) or SA-BB TPR@5%.
[1] "Membership Inference Attacks From First Principles", Carlini et al., IEEE S&P 2022.
[2] "Low-Cost High-Power Membership Inference Attacks", Zarifzadeh et al., ICML 2024.
[3] "Attention-Based Membership Inference Attacks on Graph Neural Network Through Topological Features", Guan et al., IEEE TDSC 2025.
[4] "Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture", Tang et al., USENIX Security 2022.
[5] "Topology Modification against Membership Inference Attack in Graph Neural Networks", Guan et al., Knowledge-Based Systems, 2024.
[6] "Inference Attacks Against Graph Neural Networks", Zhang et al., USENIX Security 2022.
[7] "Subgraph Structure Membership Inference Attacks against Graph Neural Networks", Wang et al., PoPETs 2024.