- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose ADAN, which uses machine learning to learn the defense noise. The method adds Gaussian noise to the gradients, and the noise scale is learned. The authors train the noise scale by optimizing the extent an gradient-based attacker would deviate from the ground truth direction, while bounding the total noise within a given level to preserve training quality. Experiments shown better performance against isotropic noise, pruning, and clipping.
Critical strengths
The idea that the defense is aware that attacks use optimization methods is interesting, many attacks are based on this framework. This trained noise clearly beats isotropic noise in their experiments, claiming effectiveness of the idea.
Critical weaknesses
- The paper claims their defense to be coordinate-wise, but based on my understanding of the paper, the training objective (7) only depends on the noise scale on that sample. Moreover, in Appendix A.2 the noise scale is normalized, if that is added to the derivation of training objective then the training objective is constant, and the training objective is 0. In this case training does not yield useful information.
- The experiment scope is very limited, and the defense is only designed against attacks based on direct optimization of gradient difference. Methods not using optimization at all (e.g. R-GAP) or regularized objectives is also not considered. The defenses benchmarked against is also very weak, only plain isotropic noise, pruning, and clipping, while many adaptive and more advanced defenses exist.
Questions for authors
- Does the method outperform advanced defenses that are adaptive (e.g. CENSOR) and defenses that does not belong to the optimization framework?
- How do we obtain coordinate-wise defense noise if the target is a function of the total noise scale? My intuition of the target is that with high noise on data points and low noise on other points, the attacker minimizing loss will be directed to low-loss region, which is low noise and away from data points. This did not really utilize fact that the defense could drive the attacker the wrong way, and the current framework seems to be a different approach.
- Appendix A.2 states that the noise is normalized, but this is not included in section 4. How is the normalization applied during training and inference? If the total noise is fixed, equation 7 appears to be constant. Does this affect the training objective?
3 comments
Answer to Question 1.
We agree that CENSOR achieves substantially higher utility than the other defense methods, but we do not consider a comparison with them to be fair. CENSOR avoids the utility–privacy trade-off by searching for a gradient that is orthogonal to the original gradient and maximizes utility. This is purchased by a very high computational cost: at each step the feasible set is the orthogonal complement of the original gradient, which is a (d−1)-dimensional set. Even if one restricts the search to a set of orthogonal basis directions, there are d−1 such directions, and evaluating them is already expensive. An exhaustive or naive search over the orthogonal complement is continuous and scales poorly with model dimension. A fair comparison should therefore account for this search cost, or compare methods under the same compute and privacy budget rather than utility alone.
Answer to Question 2.
The objective is not the total noise scale. The term related to the noise scale is the derivative of the noise’s L2 norm with respect to the input. Every coordinate of the noise is connected to the input in the computation graph, and the noise generator is built inside the leaked model. Therefore, when the attacker updates the dummy input by differentiating the gradient with respect to the input, backpropagation flows through the noise generator parameters. This derivative is affected by the entire ANAN model, which pushes the optimization of the dummy input in the opposite direction. This is the intuition behind why the defense works. See Fig. 4 for the detailed noise-generator design.
Normalizing the noise is only intended to compare privacy with the isotropic case under strictly equal utility. Normalization is multiplication by a required constant and is not included in the computation graph from the input to the noise.
Answer to Question 3.
Normalization is intended to force the noise scale to equal the constraint. Its purpose is to make the utility of the isotropic baseline and ADAN strictly comparable for comparison with the baseline, which is not a necessary step. This step must not affect the computation graph from the input to the noise; indeed, the term “normalization” is ambiguous here. What it means is that, after the noise is generated by ADAN, we multiply it by a constant chosen according to the desired strict constraint, and this constant does not affect the computation graph of ADAN. The pseudocode currently contains only the ADAN training, I will add pseudocode for the noise implementation later in the revised version.
Thank you to the authors for the response! Could the authors locate a figure version of the provided table in the revised manuscript? The tables are a bit hard to read and based on the description this does not seem to dominate other simple methods?
Q2. I understand that the objective is not the total noise scale, but my main concern is that the objective is a function of the noise scale. If the objective is a function of the noise scale, even if every coordinate do connect but their effect to the objective does not differ, they only learn through the aggregate noise scale. If so then the network only learned the scale of the noise, and not the direction of noise. After normalization the scale is normalized, and the direction remains without any training signal. This is also my main reason behind the rejection rating. Is any part of my understanding incorrect? If only the scale is learned then the direction used reduces to almost a random direction.
Q1. Even not for CENSOR specifically, there are definitely many baselines that both change the noise scale which you should compare with. (Take https://www.usenix.org/system/files/usenixsecurity25-xu-xiangrui.pdf for example). I haven't been following up on this topic for a while so I don't know the sota, but there definitely is thus a line of work using the SVD of the Jacobian to give a different noise direction. You can even test a random noise direction, the same noise changing class of defense is severely lacking.
Answer to Weakness 1.
Our method uses ADMM for constrained optimization. During optimization, the noise scale is not enforced as a hard constraint, so there is no possibility of the objective becoming invalid. Optimization proceeds in two directions: tangential and radial with respect to the noise. The noise-scale constraint restricts the radial direction, so the noise focuses more on optimizing its coordinate-wise proportions—i.e., the tangential direction. Under soft constraint, the final noise scale will not exactly equal the constraint value. Normalization is intended to force the noise scale to equal the constraint after the ADAN training by multiplying a scalar. Its purpose is to make the utility of the isotropic baseline and ADAN strictly comparable.
For more details, please refer to Answer to Questions.
Answer to Weakness 2.
Table 1. Unified quantitative comparison of defense performance across attacks. MSE ↑ indicates better privacy; Acc ↑ indicates better utility.
Analysis: On DLG/CIFAR-10, ADAN clearly lies in the upper-right region of the privacy–utility curve and outperforms the listed baselines. On GGL/ImageNet, ADAN outperforms Soteria and Isotropic, but it is not globally upper-right: Clip, Sparsify, and Quantize can match or exceed ADAN in the high-utility regime. Under GIFD, ADAN still outperforms None/Isotropic in most settings.
Additionally, as stated in the title, our proposed methods are designed for optimization-based GIA (SOTA). We will add GIFD, but R-GAP (closed-form GIA) is not included.