- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper establishes a connection between adversarial training and Wasserstein distributionally robust optimization, which enables the error analysis for a slow rate n^{-1/2} and a fast rate n^{-1} based on Restricted eigenvalue condition (a classic analysis tool in statistics). The paper mainly focus on the linear regression.
Critical strengths
-
Building a connection between two popular robust learning approaches, i.e., adversarial training and Wasserstein distributionally robust optimization. This connection is clean and insightful, as shown in Theorem 1.
-
The authors provide both rigorous statistical analysis and numerical experiments.
-
All insights.tools and technical details are well attributed and well organized, which is important in the current era.
Critical weaknesses
I do not see critical weakness for rejection.
Questions for authors
- Can the authors provide justifications for why to focus on the shrinking radius? I understand it is a standard assumption in relevant literature, but some justifications can be added to make the assumption valid.
- Can the authors discuss the possibility of extending the connection to nonlinear case?
3 comments
When we use adversarial training in practice, should we choose shrinking radius or fixed radius?
Our overall impression is that most of the times adversarial training is used in practice it focus on robustness. Hence, we would imagine that fixing the radius to a number that matches this level of robustness is the usual strategy. However, the interpretation that the radius can work as a regularizer when you shrink it with the number of samples is also applicable for a linear model, as we highlight above.
Thank you for the positive feedback. We appreciate it and try to address your questions bellow:
(Q1) Robustness versus regularization: Thank you for raising this important point. There is a dual use of the radius δ in DRO that is crucial in this paper, so we are happy to clarify this. Indeed one can see it from two different perspectives:
Robustness perspective: On the one hand, we show in our paper that DRO is connected to adversarial attacks (p=∞). So one can think of using DRO to defend the model against adversarial perturbations in input space. Here, one usually wants to guarantee δ is of a similar magnitude as the attacks we want to defend against. And hence, a larger δ might make the model more robust against such perturbations, but may also degrade generalisation in clean evaluation.
Regularization perspective: On the other hand, we connect DRO with Square-root lasso and other parameter shrinking methods. In this case, δ is a tunable regularisation parameter akin to the regularization strength. Here our results show that one obtains near-oracle performance by shrinking δ∝1/n. See also the adversarial training work [1] where the radius δ is tuned for similar purposes. We will clarify this dual use in the paper.
(Q2) Extensions to the nonlinear case: That is an interesting question. There are extensions for adversarial linear regression to kernels [2] that should extend naturally to DRO linear regression as well. In the general nonlinear case, including neural networks, such extensions might not be possible, and there are some negative results showing it [3].
References:
[1] “Regularization properties of adversarially trained linear regression,” A. H. Ribeiro, D. Zachariah, F. Bach, and T. B. Schön. NeurIPS, 2023
[2] “Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization,” A. H. Ribeiro, D. Vävinggren, D. Zachariah, T. B. Schön, F. Bach. NeurIPS, 2025
[3] “Robustness Cannot be Reduced to Regularization: Studying Adversarial Training Beyond the Linear Case,” David A. R. Robin, Rafael Pinot, Yann Chevaleyre, arXiv:2606.21488, 2026