- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper aims to design an adversarial attack to compromise fairness in node classification task for text-attributed graphs (TAGs) while at the same time preserving utility. Such fairness adversarial attacks in the existing literature mostly consider traditional graphs. To this end the authors propose Universal Fairness Poisoning Attack (USPAT), which attacks fairness by injecting malicious nodes. In the first stage of the attack, LLMs are leveraged to obtain textual features of the malicious nodes, followed by a second stage where edges are introduced. USPAT is deployed on three datasets/tasks, where it achieves better performance in terms of attack success as well as preserving utility.
Critical strengths
The paper is easy to follow and presents the ideas clearly.
It introduces a novel attack strategy, which is computationally efficient and is demonstrated to be successful in attacking fairness in TAGs.
Evaluation is performed on three datasets to demonstrate its effectiveness.
Critical weaknesses
I honestly struggle to understand the specific importance of the text-attribution setup. I see that the baselines are not as effective as USPAT, but the comparison is a bit unfair. The baseline attack strategies only poison the graph structure and not the associate text. So, the improvement in attack success rate of USPAT is not surprising.
The attack strategy is also a bit unrealistic. Consider a real-world network and an adversary. I would imagine the adversary to have an overview of the network (perhaps some text shared in one's profile) and only control the nodes that he introduces. When introducing edges, he can only do so between nodes he introduces but cannot make the nodes not in his control to form new edges. Assuming that an ordinary node need not form an edge with an adversarial node, would the attack still be equally effective?
Another important requirement for the success of a poisoning attack is its imperceptibility. It might be that the edges are introduced in such a way that it is straightforward to distinguish poisoned nodes from the real ones. The authors should consider developing tests to investigate how imperceptible are the poisoned nodes from normal nodes.
In Appendix J.2, the authors demonstrate how budget influences the effectiveness of the attack. The budget is measured in terms of the number of poisoned nodes. This investigation could be extended to also include other baselines. Given a fixed budget, how effective are the different baselines? From the settings, it is unclear if all the baseline attack strategies had similar budget.
Questions for authors
What was the assigned budget for each baseline?
How difficult is it to distinguish poisoned nodes from normal ones?
What sort of text is the LLM generating for the poisoned nodes? Are these realistic? I mean, the generated text might have semantic differences, making them easier to detect.