- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper disproves the following conjecture: the non-weakly convex definable setting, for locally Lipschitz objectives, centered omnidirectional noise with conditional fourth-moment bounded is sufficient for SGD to avoid nonminimal sharply repulsive critical points (admits an active manifold , is a local minimizer when restrict to and is Clarke critical but not limiting critical). To show this, the paper gives a counterexample on and uses a two-cycle argument to establish convergence to 0 (sharply repulsive) with probability 1.
On the other hand, the paper shows that, when the noise is not only centered and omnidirectional but also has bounded second moment, and the density is bounded, continuous, and everywhere positive, it is sufficient to avoid such critical points.
Critical strengths
- The paper disproves an open conjecture, with an explicit example. The example is also for globally Lipschitz, coercive (more than what's the conjecture requires). The proof is clearly explained: the two-cycle argument shows how bounded rescaled oscillations become convergence to 0 as step sizes shrink.
- The paper provides a complementary positive result and helps to distinguish insufficient noise assumptions from sufficient ones.
- The proof of theorem 3.1 is provided through difference quotient rescaling combined with a stationary distribution contradiction, which could be useful for studying nonsmooth dynamics.
Critical weaknesses
- The main weakness of the paper is that Theorem 3.1 only establishes avoidance to points with but not convergence of SGD. The paper also does not establish avoidance of active strict saddles beyond weak convexity in general.
Questions for authors
- Is it possible to establish escape time in Theorem 3.1, ie, the number of iterations needed to leave a chosen neighborhood of the bad point? What about convergence in Theorem 3.1?
- Is everywhere positivity a necessary condition, can it be more general?
- What are the challenges to establish avoidance of active strict saddles beyond weak convexity?