- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper characterizes the exact admissible parameter region of generalized optimistic gradient descent (GOGD) for Lipschitz operators satisfying the weak-Minty condition. The authors establish necessary and sufficient conditions for bounded iterates and square-summable increments and residuals, using a Lyapunov analysis for sufficiency and a two-dimensional linear counterexample for necessity. For monotone operators, the region also characterizes weak convergence of the iterates. Additional last-iterate convergence results are established under co-hypomonotonicity. Numerical experiments demonstrate that the enlarged parameter region can substantially reduce operator evaluations compared with previously known sufficient regions.
Critical strengths
1. Sharp characterization of the admissible parameter region. Unlike previous analyses that establish sufficient parameter conditions, this paper provides an exact characterization for a broad class of weak-Minty operators. I find the matching sufficiency and necessity results particularly valuable, as they clarify the fundamental stability limits of an existing and widely studied method.
2. Meaningful improvement over existing parameter bounds. The new region can be substantially larger than previously known regions, especially as approaches one. The theoretical improvement is supported by numerical experiments showing considerable reductions in operator evaluations on challenging quadratic minimax instances.
Critical weaknesses
1. Sensitivity to problem constants and practical parameter selection. The admissible region depends on the Lipschitz constant and the weak-Minty parameter , which may be difficult to estimate accurately in practice. Moreover, the proposed explicit parameter rule approaches the stability boundary as tends to one, potentially making it sensitive to errors in these estimates. This limits the immediate practical applicability of the enlarged region.
2. Limited evidence of practical benefits on nonlinear problems. The largest empirical improvements are obtained on structured quadratic instances closely related to the extremal linear operators used to establish the sharpness result. While these experiments effectively illustrate the benefits of the enlarged parameter region, its practical impact on more general nonlinear problems remains less clear. The paper includes a nonlinear Polar game, but this example has a relatively small , where the difference between the new and previously known parameter regions is less pronounced.
Questions for authors
-
How robust is the proposed explicit parameter rule to inaccurate estimates of and ? In particular, when is close to one, relatively small underestimations of can place the selected parameters outside the admissible region. Could the authors discuss whether a more conservative parameter rule retains most of the empirical improvement?
-
Could the authors provide additional evidence or discussion regarding the benefits of the enlarged parameter region on nonlinear problems, particularly when is relatively large? This would help clarify whether the substantial improvements observed near the theoretical stability boundary extend beyond the structured quadratic examples.
Thank you for your review. You raise important questions about parameter selection and practical robustness. We agree that the exact admissible region assumes knowledge of the problem constants, which may be difficult to estimate accurately. We have clarified this limitation in Section 4.
Q1. Sensitivity to inaccurate parameter estimates
We agree that the explicit rule (14) is sensitive when ρL is close to one: it lies close to the boundary of the admissible region and tolerates an underestimation of ρL of only order (1−ρL)2. A simple conservative alternative is to reduce α by 20% while keeping β unchanged. This retains most of the improvement: across the four high-dimensional settings, it increases the number of operator evaluations by 17–19%, and at ρL=0.9 it still requires about 15 times fewer evaluations than the best tuned parameters from the earlier GOGD regions.
In return, if L is known, the reduced rule tolerates any underestimation of ρL smaller than three-eighths of the estimated gap 1−ρL, for every estimate ρL∈(0,1). On our benchmark with true ρL=0.9, the original rule fails to converge for the tested estimates of 0.89 and below, whereas the reduced rule converges for every tested estimate down to 0.85 at essentially unchanged cost. Underestimating L is tolerated much less, so a valid upper bound on L should be used. The new Appendix C.5 gives the exact tolerances and this experiment.
Q2. Nonlinear problems with larger ρL
We have added a nonlinear Polar game with ρL≈0.89, described in Appendix D.4. On this example, tuning GOGD over our exact region requires about 17 times fewer operator evaluations than tuning over the previous region from Böhm (2023, Theorem A.1).
HPE remains faster on this example. Nevertheless, the experiment shows that the benefit of the enlarged GOGD region is not limited to the quadratic instances used to establish sharpness.
The example is still low-dimensional and structured, and we do not claim that it establishes broad practical superiority. Appendix D.4 provides the experimental setup and detailed results.