- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper develops a high-dimensional deterministic description of SGD dynamics under -adversarial training, revealing its connection to adaptive ridge regularization and explaining how adversarial perturbations influence learning-rate stability and convergence.
Critical strengths
The paper provides a rigorous and precise characterization of the entire SGD learning dynamics under -adversarial training
Critical weaknesses
-
The paper is difficult to follow due to poorly organized notation and insufficient exposition of the problem setup. For example, Equation (1) introduces several variables and functions without clearly explaining their roles. Moreover, the formulation departs from the standard presentation of adversarial training objectives in the literature, making it unnecessarily difficult for readers to recognize the underlying problem.
-
The paper is heavily driven by technical derivations and proofs, with limited discussion of the underlying insights and implications for adversarial training. Much of the analysis appears to be a direct extension of the existing homogenized SGD (HSGD) framework (Collins-Woodfin & Paquette, 2024; Paquette et al., 2022; Collins-Woodfin et al., 2024a) to adversarial training. While the theoretical results may be technically substantial, the paper does not sufficiently explain what fundamentally new understanding of adversarial training is gained from this extension. In particular, the authors should better motivate the research questions, highlight the insights behind the theoretical findings, and discuss how these findings could inform the design or improvement of adversarial training algorithms.
Questions for authors
The connection between ridge regression and adversarial training under ell2 perturbation has been well established in the literature, what different insight can be provided in this paper?
3 comments
Could you explain to me in two sentences the new insights your paper has beyond ridge regularization and HSGD?
We studied the impact of the adversarial objective on the choice of learning rate: in general, the learning rate that decreases the distance to the minimizer fastest (Polyak stepsize) combines the standard descent term with two corrections, one for the shift of the minimizer and one for the adversarial perturbation, which suggests that a schedule designed for standard training cannot be reused as is: it should also account for the adversarial perturbation, even when the minimizer is unchanged, and for the shift of the minimizer when it moves. For least squares with a single class, the loss can no longer be driven to zero for any , so the learning rate must decay, and for small the behaviour of standard training persists, with strong anisotropy making greedy schedules such as exact line search slow, a penalty that fades as grows (see the Key Takeaway of Section 4).
We thank the reviewer for their feedback. The main text was revised to address them. The results are unchanged, with one addition (Proposition 3.1). The numbers below refer to the revised PDF. For a detailed description of the changes in the revised PDF, see https://openreview.net/forum?id=MHso2fuW13¬eId=s5BtWieFDl
Weakness 1 (notation and Eq. (1)). Eq. (1) is the usual min-max objective of adversarial training, written in the notation of single-index models used in the works we build on. In the more common notation, with input x, label y, parameter θ and loss ℓ, it reads
minθE[max∥Δ∥≤δℓ(θ⊤(x+Δ),y)]+2λ∥θ∥2.
We write X for θ and a for x, and the label enters through the loss: fI(X⊤a,ϵ) stands for ℓ(X⊤a,y), where y is given by the class I (hard labels) or by a teacher model with noise ϵ (soft labels). The introduction now defines each of these symbols before Eq. (1), and derives Eq. (2) in one line: the adversary can change X⊤a by at most δ∥X∥. In Sections 1.1 and 2, each object is now introduced where it is first used, with a sentence on its role. For instance, the matrix Bi(W) collects the mean and covariance of X⊤a and the size of the perturbation. Notation not needed for the statements now appears only in Appendix B.
Weakness 2 (insights, relation to HSGD, implications).
New relative to the HSGD framework (Section 2, "Differences with prior work", previously Remark 2.1). Each update depends on the adversarial perturbation, which scales with ∥Xk∥ and is selected through an argmax. The equations must therefore also track the norm of the iterate and its overlap with the adversarial minimizer X⋆,Adv, which need not coincide with X⋆. Already for least squares, the linear Volterra equation of standard training becomes a system of coupled nonlinear ones.
New understanding of adversarial training. The introduction now poses three questions, and a Key Takeaway box in each of Sections 2–4 answers one: (i) the risk curves of SGD are predictable at non-vanishing learning rates; (ii) the ridge correspondence holds during training, with fluctuations that persist (see below); (iii) for any δ>0 the problem no longer interpolates, so no constant learning rate gives monotone decrease of the distance to X⋆, Adv, even when the minimizer is unchanged.
Implications for algorithms (Section 4). The learning rate must decay. Moreover, an adaptive schedule should account for both the shift of the minimizer (i.e. different objective) and the adversarial perturbation. Greedy schedules such as exact line search are penalized by anisotropy as in least squares for small δ, and less as δ grows (Eq. (15)). We do not propose a new algorithm: designing a practical schedule on these principles is left for future work (Section 5).
Question (insight beyond the known ridge connection). The known results are static: under appropriate assumptions, they identify the minimizer of ℓ2-adversarial least squares with a ridge regression solution (Xing et al., 2021; Ribeiro et al., 2023). To our knowledge, the correspondence had not been characterized along the trajectory of SGD. Section 3 shows that it extends there, through the ratio q(Xk)=∥Xk∥/2R(Xk), in three ways.