- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether graph-dependent feedback can remain unchanged from a GNN judge’s perspective while becoming harmful for an LLM trained on the underlying text. The attacker edits node text and graph edges while preserving all discrete selection and labeling fields and keeping continuous transcript fields within fixed tolerances. The proposed framework combines a second-order attack construction, a diagnostic for correctability and student exposure, and a defense that reweights feedback under class-wise coverage constraints. Experiments span five text-attributed graph datasets and two feedback pipelines over five seeds. The reported results show improved attack feasibility over matched first-order controls and reduced downstream harm under the proposed defense.
Critical strengths
The threat model isolates a meaningful failure mode. Preserving selected IDs, labels, memberships, and continuous decision fields is substantially more specific than merely preserving prediction accuracy. The paper also explicitly excludes the raw text from the invariant record, which makes the claim interpretable.
The matched controls and mechanism ablations are useful. On Cora, adaptive FC-CEFA increases pre-cap feasibility from 51.4% to 90.0% and unnormalized harm from 0.117 to 0.221 over the first-order control (Table 1). Table 2 separately removes mixed-curvature correction and graph-side Newton refinement. These comparisons provide more direct evidence for the construction than the failures of native attacks that were never optimized for transcript preservation.
Critical weaknesses
The central empirical claim depends on a discrete realization step that is not sufficiently explained. Theorem 5.2 establishes a local result for relaxed states; sparse binary edge edits and actual token substitutions need not retain that cancellation. Appendix A.3.1 describes a top-budget projection and tokenizer-aware beam search, but does not specify the text relaxation, candidate construction, or how the graph correction is recovered after discretization. Tables 1–2 nevertheless report worst-case transcript drift near (10^{-15}) with high acceptance rates. The paper states that the complete gate is checked, but a concrete realized example and an independently recomputed fieldwise transcript would make this unusually strong result much easier to assess. This matters because discrete feasibility is the main bridge from the geometric argument to the claimed vulnerability.
The defense evaluation does not sufficiently specify the adaptive optimization. CT-CAFT suppresses exposure over a tube generated by the same attack family used for evaluation. The paper acknowledges this scope, but it is unclear how attacks are regenerated as defensive weights change, whether gradients account for the defense, and how the inner search is checked for convergence. Theorem 5.3 does not resolve this concern: it assumes an independent student-update approximation bound (\xi_t), which is not numerically established. Consequently, the reported 52.2–60.0% harm reduction is promising, but the current description leaves uncertainty about how strongly the defense was challenged.
Questions for authors
Could you provide one successful discrete attack with the original and edited text, exact edge toggles, and a fresh recomputation of every transcript field? Please clarify whether the judge’s text features are recomputed from the edited raw text and explain how the reported near-machine-precision agreement survives discretization.
What is the precise adaptive attack procedure against CT-CAFT? Are continuation states and directions regenerated after defensive weight updates, does optimization differentiate through or otherwise account for the defense, and does increasing the search budget materially change the reported harm reduction?
In the 20-round co-teaching setting, is each attacked transcript compared against an independently evolving clean trajectory, or against an unedited evaluation at the current attacked model state? How are the token and edge budgets accumulated across rounds when computing IFH?