- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Propagation Identity Perturbation (PIP), an adversarial threat model for social event / information diffusion forecasting, where the attacker replaces up to B participant identities in an observed cascade while keeping timestamps and the social graph fixed. Replacements are restricted to “behaviourally plausible” candidates (active within ρ days, similar relative position in cascades, within r hops of the previous participant). The attack is a HotFlip-style greedy search on a source model (MILD): gradient scores propose candidates, the top 128 are verified by exact forward passes, and the best substitution is accepted each round. The resulting perturbed cascades are transferred, without queries, to five target models (RotDiff, DyHGCN, MS-HGAT, DisenIDP, GODEN) on one Twitter dataset. With B = 6, average Hits@10 drops from 34.02% to 25.29% (relative drop 25.66%), and five adversarial-training-style defenses reduce this only slightly (best: AT, 22.75%). The paper also gives four theorems on search space reduction, transferability, monotonic ascent and a loss bound.
Critical strengths
A new and relevant attack surface. Most graph robustness work perturbs edges or node features, while this paper looks at who participates in a cascade, which is the main input of diffusion forecasting models. The results show that these models are sensitive to a small number of identity changes across five quite different architectures. This matters to my assessment because it identifies a vulnerability that has received little attention in diffusion forecasting.
Critical weaknesses
- The defense evaluation does not support the claim that existing defenses fail.
Table 2 only reports defended models on perturbed test data. The clean accuracy of each defended model is not reported, and RHD for the defended rows is still computed against the undefended model’s clean accuracy. Without both clean and robust accuracy after defense, we cannot judge what AT (or any defense) actually costs or gains. Only one budget (B = 6) is reported for the defenses. A defense evaluation should show results under several budgets, and ideally with the defense trained and tested under different budgets. The defenses are trained on a fixed set of perturbations generated once on MILD, not regenerated against the model being trained, so “AT” here is static adversarial data augmentation rather than Madry-style adversarial training. Training is also very light (at most 10 epochs, learning rate 2×10⁻⁵), and epoch 0 is allowed as a no-update fallback. Several defended numbers are identical to the undefended ones to four decimals (MS-HGAT with MART and ATFL: 23.3667; DisenIDP with SemRoDe-MMD: 23.8488), which suggests that no update was selected in those cases. All five defenses belong to the adversarial-training family; no detection-based or graph-specific defenses are evaluated, although the paper claims PIP is “unaddressed by standard graph defenses”. The budget is defined as an absolute number of positions (B = 6), but the cascade length distribution and the fraction of positions actually changed are not reported. Changing 6 users in a short cascade is very different from a long one; a ratio of the cascade length would be a more meaningful budget.
- The experiments are limited, and the novelty and theory are overstated.
Only one dataset (Twitter, 346 test cascades) is used. The ablation against random legal replacement is only on MS-HGAT at B = 6, and the budget sensitivity (Table A2) is reported on the source model MILD itself, not on the transfer targets. There are no other attack baselines. The plausibility of the perturbations is never tested. The constraints appear loose (on average about 2,900 of 12,627 users are eligible per position, and results do not change for ρ ≥ 3 days), and the attack deliberately picks the least expected substitutions, so a simple likelihood-based detector might catch them. Substituting elements of an ID sequence with gradient-guided search has been studied in sequential recommendation (e.g., substitution-based profile pollution attacks and their defenses, Yue et al., 2022), which is not discussed. The claim of being the first threat model of this kind needs to be positioned against this work. The theory adds little. Theorems 1 and 3 hold by construction. Theorem 2 assumes the source and target representations are aligned up to an orthogonal matrix, but Appendix A.2.3 states that the target models share no common latent dimensions or coordinate system, and the proof of Theorem 2 uses “W is diagonal” instead of orthogonal.
Questions for authors
- Please report the clean Hits@10 of every defended model, and which checkpoint (including epoch 0) was selected for each defense. Are the identical defended and undefended results due to the epoch-0 fallback?
- How do the defenses perform when adversarial examples are regenerated against the model being trained, with properly tuned training, and under several budgets?
- What is the cascade length distribution, and how many positions are actually changed per cascade? Can the budget be expressed as a fraction of cascade length?
- How detectable are the perturbed cascades, e.g., using the clean model’s likelihood of each participant given the prefix?
- In what realistic scenario can an attacker replace (rather than add) participants in an observed cascade, given that platforms keep logs of who actually reshared?
- How does Theorem 2 apply when, as Appendix A.2.3 states, the targets share no latent coordinate system with the source?
6 comments
Reply to Reviewer [2/3]
4. Behavioral plausibility, likelihood detection, and filtering.
The new experiments directly test the proposed likelihood detector. With genuine participants from the same edited positions as negatives, target-model conditional-likelihood scores achieve AUC 0.72–0.80 on PIP, versus 0.84–0.90 on random feasible substitutions (Table A26). At 5% false positives, they detect 12.2–22.5% of PIP edits. These results quantify detectable structure while showing that likelihood-based rejection leaves most PIP edits undetected at that operating point.
We also evaluate detection as an actual defense: thresholds are calibrated on clean validation data, flagged observations are removed, and attacked accuracy is compared with the corresponding filtered clean accuracy. At nominal 5% false positives, mean RHD decreases from 25.84% to 21.24%; the filter flags 41.1% of edited identities, has 6.7% realized test false positives, and removes 1.46 genuine observations per clean cascade (Table A28). Under independently generated label-blind prefix attacks, filtering leaves 14.59–20.30 percentage points of target Hits@10 loss (Table A29). The evaluation thus extends beyond adversarial-training objectives to an observation-level detection and filtering defense.
Behavioral checks also support the constraints. PIP replacements have a 4.1% observed predecessor-transition rate, compared with 0.4% for random feasible replacements, and a smaller mean distance to recorded activity, 0.62 versus 0.67 days (Table A20). Regenerating attacks with a one-day activity window yields 25.47% mean RHD. Requiring one-hop substitutions under the defined transition-plus-social-neighbor relation yields 20.16% (Table A19). The vulnerability persists under substantially tighter candidate rules.
5. Dataset breadth, competitive searches, and the contribution.
The revised experiments include all 349 Douban test cascades. Mean RHD increases from 12.51% at to 44.47% at , with a 95% interval of [41.9%, 47.0%] for the latter. The nominal 3% length-proportional budget gives 15.06% mean RHD (Table A12). In label-blind endpoint evaluation, using each of six architectures as the source on both datasets gives positive Hits@10-decrease estimates in all 60 cross-architecture comparisons, with 56 intervals excluding zero (Tables A23–A24).
Search comparisons now cover random proposals with exact verification, gradient-only substitution, fourfold candidate expansion, and a rank-10 margin objective (Tables A13–A17). PIP achieves 25.84% mean RHD using 320 exact source candidate evaluations per cascade, compared with 25.43% and 1,154 evaluations for random proposals with exact verification (Table A14). This is a 72.3% reduction in exact candidate evaluations at comparable aggregate damage. Matched per-step candidate caps further show a consistent gradient advantage on MS-HGAT, with gains of 5.54–8.86 RHD points and positive paired intervals (Table A15).
Yue et al. (2022) study gradient-guided item substitutions in interaction histories. HotFlip is also a relevant optimization precedent. PIP's contribution is the explicitly constrained robustness problem for identity-bearing diffusion records: a participant-position budget, fixed timestamps and background graph, training-derived activity/role/reachability constraints, and transferable attacks evaluated across forecasting architectures. The revised analysis also formalizes the identity-to-incidence representation of this problem (Appendix B.1) and cites such relevant references, making its connection to constrained graph rewiring precise.
6. Attacker control and the unchanged prediction label.
Sections 3.1–3.2 define an input-record integrity threat. A concrete setting is a forecasting service consuming a cascade feed through a compromised collection, identity-mapping, or preprocessing component: the adversary controls selected identity fields in the record delivered to the predictor. The platform's realized reshare history supplies the evaluation label, while the downstream input record is corrupted. Features derived from the edited prefix are recomputed consistently, and timestamps and the background social graph remain fixed.
The revision additionally evaluates attacks whose generator sees only the observed prefix. The next participant is available exclusively to the evaluator. Endpoint entropy attacks decrease Hits@10 on all five targets in both datasets, with positive 95% intervals (Table A22). Independent attacks at all 7,883 Twitter prediction prefixes, with a fresh six-edit budget for each prefix, cause 16.44–24.36 percentage points of target accuracy loss across the two label-blind objectives (Table A25). These experiments directly establish transfer damage without future-label access.
Reply to Reviewer [3/3]
7. Theorem 2, heterogeneous representations, and prediction damage.
Theorem 2 now uses a general linear map and , allowing different representation dimensions. Its endpoint-alignment condition gives
The revised proof uses the reverse triangle inequality and operator norm. This is an explicit conditional displacement bound. Theorems 1 and 3 characterize the feasible search space and the acceptance rule. The connection to target loss is developed through directional gain, readout behavior, and finite-edit terms in Theorem 4 and Appendix B.2.
Appendix B.2 further works in the shared output coordinates of participant identities, which are available even when hidden-state dimensions differ. Proposition 8 provides an exact transported-loss identity and a sufficient condition for excluding the true participant from the target's top- set. The prediction-level conclusion is supported directly by the expanded Hits@10 experiments above.
Reply to Reviewer [1/3]
Thank you for identifying the importance of participant identities as an attack surface. The revision addresses the evaluation questions with clean/attacked results for all 25 baseline target-defense combinations, stronger online training, absolute and length-proportional budgets, a second dataset, likelihood detection and filtering, and label-blind cross-architecture experiments. These additions establish both the practical scope of PIP and the accuracy losses it produces under substantially broader evaluation.
1. Clean utility, robust accuracy, and checkpoint selection.
The revised Table A6 reports clean and attacked Hits@10 for all 25 target–defense combinations, with each relative hit decrease (RHD) computed against that checkpoint's own clean accuracy:
RHD=100H@10cleanH@10clean−H@10attacked.
For example, AT on DyHGCN changes clean/attacked accuracy from 33.16%/25.38% to 33.41%/26.21%, yielding 21.56% RHD. On DisenIDP, the corresponding change is 34.52%/23.85% to 35.13%/25.71%, yielding 26.80% RHD. The table therefore separates clean utility from robustness recovery. Across the baseline defended checkpoints, clean accuracy changes by less than one percentage point, while RHD remains 21.56–30.91%.
The identical baseline entries contribute zero measured robustness gain. Checkpoint selection uses validation data and includes epoch 0. Table A10 marks the initial selections for the stronger-training comparisons: MS-HGAT with TRADES and MART, RotDiff with MART, and GODEN with TRADES. These entries represent selection of the initial model and are interpreted accordingly. The table separately presents the retained trained checkpoints and their clean/attacked outcomes.
2. Online adversarial training, stronger optimization, and training–test budget transfer.
The original baseline uses fixed source-generated training pairs. The expanded evaluation includes online white-box AT, which regenerates adversarial training inputs against the model being trained (the revised Tables A8–A9). Table A8 increases the learning rate from 2×10−5 to 2×10−4 or 10−3 and extends training to at most 20 epochs, with patience five. At test budget B=6, tuned AT reduces RHD to 9.20% on DyHGCN, 9.54% on MS-HGAT, 11.97% on DisenIDP, 16.12% on GODEN, and 21.12% on RotDiff. Thus stronger training produces substantial architecture-dependent recovery, and the revised conclusions reflect that result.
Table A10 also strengthens TRADES and MART and reports their clean-accuracy costs. For DyHGCN, tuned AT reaches 30.04% attacked accuracy with 33.08% clean accuracy. Tuned MART reaches a smaller 6.22% RHD, but its clean/attacked accuracies are 27.72%/25.99%. Reporting both quantities makes the trade-off explicit: the smallest relative drop need not provide the best attacked accuracy.
The defense comparison now spans B=1,…,6 and four length-proportional budgets (Table A7). Online AT is also trained separately at Btrain=2 and 6 and evaluated at multiple test budgets (Table A9). at Btest=6, their RHD differs by at most 0.54 percentage points under the baseline schedule. The transfer evaluation uses the same frozen, target-independent test perturbations for the compared checkpoints. Training-time online optimization and test-time transfer evaluation are therefore reported as distinct experimental components.
3. Realized edits and length-proportional budgets.
Table A11 reports the number and share of positions actually changed. On Twitter, the nominal B=6 attack makes 1,645 substitutions across 346 cascades: 4.75 edits per cascade on average and 20.0% of all recorded positions. At B=1, the realized share is 4.2%, and mean target RHD is 5.87%.
We additionally evaluate the per-cascade budget min(6,⌈ϵL⌉) for ϵ∈0.03,0.05,0.10,0.20. For nominal ϵ=3, integer rounding produces 479 substitutions, or a realized 5.8% of positions, and 11.07% mean RHD. All five target-specific 95% cascade-bootstrap intervals are positive. Reporting realized shares alongside the nominal ratio makes the effect of short cascades and integer rounding directly visible.