- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
In this paper, the authors analyze a 2-layer rank-1 linear model with logistic loss, proving that one needs at least two 2 datapoints to create a minimal example that would display the edge-of-stability phenomenon observed by Cohen et al. (2021) for real neural networks. The primary motivation for doing this seems to be in connection to Kalra et al. (2025) which showed that 1 data point is sufficient to reproduce the phenomena in the regression setting. In other words, this paper shows you need 2 data points for the 2-class classification setting. In the zero-margin limit, they also make a connection to Neimark-Sacker bifurcation to show perpetual loss and sharpness oscillations for their model.
Critical strengths
The paper is clearly written, the assumptions are presented honestly, and the exposition is easy to follow.
Critical weaknesses
I am not sure about the theoretical value of this paper on such a simplistic setting beyond "it is a sequel to Kalra et al. (2025) but now for classification and it shows that you need at least two datapoints for classification."
Questions for authors
Since my main concern for this paper is it's scope and relevance (rather than content or clarity), I ask the following two questions:
- What insights from this theory could generalize to used-in-practice neural networks once the assumptions of the toy model are nolonger true? (That is not already covered by the current EoS literature.)
- Alternatively, what practical recommendations about neural network practice that would not proposed by the current literature does this suggest?
2 comments
I will retain my score of 2. For reason that the authors' responses come down to their opinion is different than mine, but none of the concrete concerns I have are misunderstandings. In particular, my original concern is that this model is highly stylized and holds limited relevance to used-in-practice networks. The authors rebuttal states that the authors disagrees and the authors' opinion is that 4-layer MLP is sufficient to prove relevance to practical networks. I do not agree with this opinion.
Similarly, for the comparison with Kalra et al. (2025) my understanding is that the authors start with the premise Kalra et al. (2025), change the setting slightly, and prove new phenomenon in that setting. The authors rebuttal lists some of the new phenomenon they prove. Thus, we are again at simply a difference in opinion on whether this constitutes a significant/relevant contribution to the literature.
Thus, I will retain my score and leave the final decision to the AC after making my premises clear above.
Thank you for your thoughtful comments.
Critical Weakness 1: I am not sure about the theoretical value of this paper on such a simplistic setting beyond "it is a sequel to Kalra et al. (2025) but now for classification and it shows that you need at least two datapoints for classification."
While our work does use a similar dynamical systems approach as Kalra et al., we note that an the sufficient condition we show for the two datapoint asymptotic setting shows loss and (approximate) sharpness oscillations that can be quasi-periodic or periodic with a period greater than 2. Compared to many prior regression works that focus on period-2 oscillations, we provide an analysis of a setting where looking at a two-step update would be insufficient.
Question 1: What insights from this theory could generalize to used-in-practice neural networks once the assumptions of the toy model are no longer true? (That is not already covered by the current EoS literature.)
Consider Figure 1 from our paper, where we show Edge of Stability for a 4-layer MLP. As the dynamics do not reflect a period-2 orbit, our analysis provides a potential explanation for the source of such intricate dynamics via the Neimark-Sacker bifurcation, where dynamics on the invariant curve can be aperiodic or periodic.
Question 2: Alternatively, what practical recommendations about neural network practice that would not proposed by the current literature does this suggest?
Our work does have immediate practical implications.