- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors introduce a white-box membership inference attack that uses features from self attention maps instead of relying on the model outputs. The authors define two feature representations: 1) Transitional features: that measure the KL divergence between attention distributions of two adjacent layers, and 2) perturbation features: that capture how attention shifts when inputs are edited. They train a classifier on these features and find that it performs better than existing output-based MIA attacks.
Critical strengths
- The proposed MIA is pretty novel. Most attacks rely on output probabilities or signals. Using self attention to detect membership information is interesting and novel.
- The methodology to test for generalization is rigorous: The authors do a systematic controlled study by doing target-blind leave one domain out evaluations. This essentially shows that the attention-based membership signal transfers to unseen domains rather than just overfitting to specific training data.
- The ablations and feature analysis are well reported, such as the contribution of different layers and perturbation methods.
Critical weaknesses
- The paper empirically shows that the attention distributions differ for members and non-members but it does not isolate whether there are any specific factors (e.g., specific circuit types) that are driving the MIA signal.
- The other concern I have is about the dimensionality overhead: extracting attention matrices across all layers and heads can be massive, as compared to the output-based MIA attacks. This definitely raises questions on the practicality of this attack.
Questions for authors
- Did you find any specific mechanistic factors driving the MIA signal? also did you run any ablations to check if any specific attention heads disproportionately contribute to the perturbation and transitional signals?
- Can you compare the scalability of the proposed MIA with output-based attacks, as I feel the output-based attacks would be much cheaper to run?
Thanks for the additional information. After reviewing the response, I keep my current review score.