- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes Objective-Prescribed Escape Gradient Descent (OPE-GD), a deterministic mechanism for escaping local minima in non-convex optimization. Upon detecting stagnation, the method prescribes a target objective value and solves a scalar nonlinear equation to determine an escape step along the gradient direction. The algorithm then resumes standard gradient descent. Numerical experiments on one-dimensional non-convex functions demonstrate improved success rates in reaching global minima compared with standard gradient descent.
Critical strengths
-
The proposed mechanism is conceptually simple and can be integrated into standard gradient descent without requiring second-order information.
-
The numerical examples illustrate how occasional large steps can facilitate transitions between attraction basins.
Critical weaknesses
-
Eqs. (3) and (4) incorrectly claim that for . This is false when , as occurs in the quartic example used in the experiments. Moreover, the prescribed dynamics are not invariant under constant shifts of the objective function. These issues undermine the claimed monotonic decrease of the objective.
-
Strategy 2 (Eq. (10)) selects the admissible root that minimizes the objective value. However, all exact roots satisfy , making their objective values identical. Consequently, this criterion cannot meaningfully distinguish between admissible roots.
-
The method cannot escape exact stationary points, since its updates remain proportional to the gradient. More generally, the paper provides no theoretical conditions guaranteeing the existence of admissible roots, successful escape from local minima, or convergence. The effectiveness of the proposed escape mechanism therefore remains theoretically unsubstantiated.
-
The experiments are limited to one-dimensional synthetic functions, where the step-size search explores an interval of candidate points. In higher dimensions, the search remains restricted to a single line, raising questions about the method's general applicability. Moreover, the numerical root-search procedure introduces potentially substantial computational overhead, which is not accounted for in the comparison with standard GD. No higher-dimensional experiments or comparisons with existing escape mechanisms are provided.
Questions for authors
-
How is the monotonic decrease in Eq. (4) justified when ?
-
How does Strategy 2 distinguish between admissible roots when they all yield the same objective value?
-
Under what conditions can the method guarantee successful escape, particularly at stationary points where ?