- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents an empirical audit of privacy risks in LLM-enhanced graph neural networks for text-attributed graphs. It introduces a five-stage framework covering dataset preparation, victim model training, privacy attacks, risk assessment, and defense analysis. The study considers six main datasets, six attacks targeting link existence, node labels, and training-set membership, and 42 configurations combining six feature enhancers with seven GNN backbones. The main experiments compare semantically enriched representations against shallow text features, reporting improved node classification performance alongside increased attack performance. The authors also examine embedding-distance and prediction-confidence distributions and evaluate embedding-, link-, and label-level differential privacy mechanisms. The main contribution is a systematic empirical investigation of the relationship between representation enhancement, predictive utility, and privacy vulnerability.
Critical strengths
-
The paper addresses a practically important privacy question in an increasingly relevant graph-learning setting. Integrating language-model representations into GNNs can improve predictive performance, but its implications for privacy deserve explicit evaluation. The paper investigates this issue across link, label, and membership inference rather than considering utility alone. This makes the study relevant to practitioners deciding whether and how to adopt semantically enriched graph representations.
-
The paper connects utility, attack performance, and defensive costs within a coherent evaluation framework. Table 2 and Figures 3–5 jointly present classification performance and several forms of privacy vulnerability. Figure 6 offers an intuitive examination of representation and confidence separation, while Figures 7–8 illustrate the utility costs associated with privacy perturbations. This combination provides a useful empirical perspective on the consequences of representation enhancement.
Critical weaknesses
-
The label-inference evaluation requires a clearer operational definition and comparison baseline.The label-inference evaluation needs clarification. HGA’s neighboring-label source is unspecified, and its accuracy varies across feature enhancers (Fig. 4), although it should remain unchanged with fixed graph structure, target nodes, and observed labels. NIA also requires comparison with direct posterior argmax on identical target nodes to establish whether its results reveal privacy leakage beyond improved classification.
-
The defense curves do not sufficiently establish a reduction in exploitable privacy leakage. Figures 8(a–b) report attack AUCs below 50%, which become above-chance after reversing score orientation. Although Section 5.6 specifies a fixed MLA attack, its reduced AUC does not establish reduced privacy leakage. The privacy–utility conclusions therefore require score orientation selected on independent validation data and evaluation of attacks adapted to the defended outputs.
Questions for authors
-
About HGA: In Section 4.3, does HGA use observed ground-truth labels or predicted neighboring labels? Which graph, target nodes, and label information are held fixed across feature enhancers? How does this explain the representation-dependent HGA results in Figure 4?
-
Under the access assumptions in Section 3, can the attacker directly query the target node’s class posterior? If so, how does NIA compare with the direct posterior-argmax baseline on exactly the same target nodes? If not, what restriction prevents this query?
-
As for the defense interpretation: In Figures 8(a) and 8(b), is MLA’s score orientation fixed across privacy budgets? Do the reported privacy–utility conclusions remain valid when the attacker selects score orientation using independent validation data and adapts the attack to each defense setting?
3 comments
Rebuttal by Authors (Part 2/2)
Q3: MLA Score Orientation and Defense Interpretation
Thank you for this insightful question. Yes, MLA uses a fixed score orientation across all privacy budgets in Figs. 8(a) and 8(b). As described in Section 4.3 and Appendix A.3, smaller Manhattan distances between node posteriors indicate a higher likelihood of a link, corresponding to the score . We agree that an AUC below 50% indicates a reversed rather than eliminated ranking signal: an attacker who reverses the orientation obtains , which is above chance. Therefore, the reduced AUC in Figs. 8(a) and 8(b) reflects the reduced effectiveness of the fixed MLA attack, and should not be interpreted as reduced exploitable leakage at small privacy budgets.
To avoid ambiguity, we will revise the corresponding discussion in Section 5.6 and Appendix A.3 as follows:
MLA uses a fixed score orientation, s_uv=-||R_u-R_v||_1, across all privacy budgets. The AUC values in Figs. 8(a) and 8(b) therefore measure the effectiveness of this fixed attack, rather than the maximum inference capability of an adaptive attacker. When the AUC falls below 50%, the ranking signal is reversed rather than eliminated, and an attacker who selects the score orientation on validation data, or adapts the attack to the defended outputs, may still achieve above-chance link inference. Accordingly, the observed decrease in fixed-MLA AUC indicates reduced effectiveness of the evaluated attack, and does not by itself establish reduced privacy leakage. Evaluating DP defenses against such adaptive attackers is an important direction for future work.
We sincerely thank the reviewer again for the thoughtful and constructive suggestions, which have helped us improve the clarity and rigor of our privacy evaluation.
Looking Forward to Your Feedback
Dear Reviewer zbJb,
We sincerely appreciate the time and effort you have devoted to reviewing our submission. Your comments have been very helpful, and we have carefully addressed each of them in our response.
With the discussion period nearing its end, we would be grateful to hear whether our response has resolved your concerns. If any points remain unclear, we would be glad to provide further clarification.
Thank you very much for your consideration.
Best regards,
The Authors
Rebuttal by Authors (Part 1/2)
Thank you for your careful and constructive feedback. We particularly appreciate your recognition of the practical importance of investigating privacy risks in LLM-enhanced GNNs and the value of jointly evaluating predictive utility, privacy vulnerabilities, and defense costs. Below, we address your three questions in detail.
Thank you for this important question. We clarify that our HGA implementation uses a combination of observed ground-truth labels and model-predicted neighboring labels. For neighboring nodes with available training labels, the attacker directly uses their observed labels. For the remaining neighbors, the attacker queries the victim GNN and obtains predicted labels through argmax over their class posteriors. The target node's label is then inferred through majority voting.
Across different feature enhancers, we keep the graph structure, target nodes, data splits, and ground-truth labels unchanged. Consequently, the observed neighboring labels remain fixed, while the predicted labels of unlabeled neighbors vary with the learned node representations. These differences affect majority voting and explain the representation-dependent HGA accuracy in Fig. 4. Since only 10% of nodes are used for training, most neighboring labels are model-predicted, making HGA performance sensitive to the victim model's predictions.
To avoid ambiguity, we will clarify the hybrid-label HGA implementation in Section 4.3 and Appendix A.3 as follows:
Thank you for raising this important concern. Yes, under the black-box access assumptions in Section 3, the attacker can directly query the target node's class posterior. We agree that this makes Direct Posterior-Argmax a necessary baseline: it predicts the target label as the class with the highest posterior probability, without modifying the graph. In contrast, NIA injects an attacker-controlled node connected to the target and infers the label from the resulting posteriors, with additional sensitivity-based scoring when prediction confidence is insufficient.
We compare Direct Posterior-Argmax, HGA, and NIA under GCN on Cora (Table 2 and Fig. 4):
NIA consistently underperforms Direct Posterior-Argmax in this setting. We therefore do not claim that NIA reveals label information beyond direct posterior access; rather, it represents a complementary active attack involving additional adversarial capabilities.
Importantly, Direct Posterior-Argmax is itself a label-inference attack: because target labels are private, the higher accuracy of LLM-enhanced GNNs directly increases how accurately a querying adversary can infer them. We acknowledge that this label exposure is closely tied to predictive utility. Our matched-utility analysis (Appendix B.10.2, Table 13) further shows that at closely matched utility, where direct posterior exposure is comparable, Linq-GCN still yields 2.41 percentage points higher label-inference accuracy than Shallow-GCN on Reddit, although this difference should be interpreted in light of the variability across runs.
We will revise Section 5.4 accordingly, adding Direct Posterior-Argmax as a reference baseline evaluated on identical target nodes, and describing NIA as an active attack under a complementary threat model rather than an inherently stronger label-inference method. The access assumptions in Section 3 remain unchanged.