Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability
Abstract
We study logistic regression on linearly separable data under gradient descent with a large constant stepsize . Such dynamics may exhibit a characteristic Edge of Stability phenomenon, in which the loss initially oscillates before transitioning to a stable phase of monotone decrease. Existing work provides a tight bound in dimension as and conjectures a bound independent of in arbitrary dimensions . In this paper, we disprove this conjecture by showing that, for every fixed sample size and sufficiently small margin , the worst-case transition time is uniformly over . The key challenge in establishing a tight bound is that the sample contributing most strongly to the gradient can change repeatedly across iterations. To address this issue, we control such changes by induction on dimension and sample size, and construct matching hard instances.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.